[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:03:10 dist-test-slave-sb2z krb5kdc[596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:03:10 dist-test-slave-sb2z krb5kdc[596](info): set up 2 sockets
Aug 09 12:03:10 dist-test-slave-sb2z krb5kdc[596](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 12:03:11.702389 589 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:03:11 dist-test-slave-sb2z krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:03:11.920543 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38861
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:38861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:12.530485 619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:12.531031 619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:12.531286 619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:12.541751 619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:12.542012 619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:12.542188 619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:12.542383 619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:12.542536 619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:12.556700 619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:38861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:12.558856 619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:12.560874 619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:12.584522 624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:12.584726 625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:12.584610 630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:12.587512 619 server_base.cc:1034] running on GCE node
I20260809 12:03:12.588752 619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:12.590950 619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:12.592252 619 hybrid_clock.cc:648] HybridClock initialized: now 1786276992592096 us; error 80 us; skew 500 ppm
I20260809 12:03:12.592885 619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:12 dist-test-slave-sb2z krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276992, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:12.597870 619 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:03:12.600797 619 webserver.cc:460] Webserver started at http://127.0.147.126:37787/ using document root <none> and password file <none>
I20260809 12:03:12.601881 619 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:12.602090 619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:12.602622 619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:12.607607 619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "2cef94cf55d9464b9195226df8230e23"
format_stamp: "Formatted at 2026-08-09 12:03:12 on dist-test-slave-sb2z"
server_key: "dec32646023748fb8b0ebc3481de7f6c"
server_key_iv: "27f0b23243ef552a51c8ad8ae302bdf5"
server_key_version: "encryptionkey@0"
I20260809 12:03:12.608566 619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "2cef94cf55d9464b9195226df8230e23"
format_stamp: "Formatted at 2026-08-09 12:03:12 on dist-test-slave-sb2z"
server_key: "dec32646023748fb8b0ebc3481de7f6c"
server_key_iv: "27f0b23243ef552a51c8ad8ae302bdf5"
server_key_version: "encryptionkey@0"
I20260809 12:03:12.617540 619 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260809 12:03:12.624225 638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:12.626103 619 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260809 12:03:12.626420 619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "2cef94cf55d9464b9195226df8230e23"
format_stamp: "Formatted at 2026-08-09 12:03:12 on dist-test-slave-sb2z"
server_key: "dec32646023748fb8b0ebc3481de7f6c"
server_key_iv: "27f0b23243ef552a51c8ad8ae302bdf5"
server_key_version: "encryptionkey@0"
I20260809 12:03:12.626719 619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:12.645450 619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:12.651839 619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:12.652343 619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:12.700973 619 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:38861
I20260809 12:03:12.701169 713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:38861 every 8 connection(s)
I20260809 12:03:12.703624 619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:12.705096 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 619
I20260809 12:03:12.705591 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:12.708755 589 external_mini_cluster.cc:1428] Setting key f4e90c6c281d62d1a124961eabf45546
I20260809 12:03:12.711668 717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:12.734616 717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23: Bootstrap starting.
I20260809 12:03:12.741628 717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:12.744112 717 log.cc:824] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:12.749763 717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23: No bootstrap required, opened a new log
Aug 09 12:03:12 dist-test-slave-sb2z krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:12.763146 717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cef94cf55d9464b9195226df8230e23" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38861 } }
I20260809 12:03:12.763902 717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:12.764139 717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cef94cf55d9464b9195226df8230e23, State: Initialized, Role: FOLLOWER
I20260809 12:03:12.765288 717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cef94cf55d9464b9195226df8230e23" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38861 } }
I20260809 12:03:12.765740 717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:12.766036 717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:12.766342 717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:12.770040 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:12.715039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.0.147.126:38861)
0809 12:03:12.720544 (+ 5505us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:12.720563 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:12.720682 (+ 119us) server_negotiation.cc:407] Connection header received
0809 12:03:12.735252 (+ 14570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:12.735276 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:12.735970 (+ 694us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:12.736425 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:12.741148 (+ 4723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:12.743581 (+ 2433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:12.747438 (+ 3857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:12.748630 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:12.749092 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:12.757487 (+ 8395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:12.757576 (+ 89us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:12.757612 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:12.757674 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:12.762938 (+ 5264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:12.764087 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:12.764099 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:12.764105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:12.764446 (+ 341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:12.765049 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:12.765054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:12.765061 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:12.766017 (+ 956us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:12.766485 (+ 468us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:12.767476 (+ 991us) server_negotiation.cc:299] Negotiation successful
0809 12:03:12.768099 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2571,"thread_start_us":2075,"threads_started":1}
I20260809 12:03:12.770298 717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cef94cf55d9464b9195226df8230e23" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38861 } }
I20260809 12:03:12.771651 717 leader_election.cc:302] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cef94cf55d9464b9195226df8230e23; no voters:
I20260809 12:03:12.773464 717 leader_election.cc:288] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:12.775717 725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:12.777871 717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:12.778839 725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [term 1 LEADER]: Becoming Leader. State: Replica: 2cef94cf55d9464b9195226df8230e23, State: Running, Role: LEADER
I20260809 12:03:12.780491 725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cef94cf55d9464b9195226df8230e23" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38861 } }
I20260809 12:03:12.791591 727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cef94cf55d9464b9195226df8230e23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cef94cf55d9464b9195226df8230e23" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38861 } } }
I20260809 12:03:12.793016 727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:12.792419 728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cef94cf55d9464b9195226df8230e23. Latest consensus state: current_term: 1 leader_uuid: "2cef94cf55d9464b9195226df8230e23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cef94cf55d9464b9195226df8230e23" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38861 } } }
I20260809 12:03:12.793934 728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:12.804661 738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:12.817637 738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:12.836450 738 catalog_manager.cc:1347] Generated new cluster ID: 28ea469c55c04a56a4b325a2806c77f4
I20260809 12:03:12.836642 738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:12.871169 738 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:12.872977 738 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:12.888825 738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cef94cf55d9464b9195226df8230e23: Generated new TSK 0
I20260809 12:03:12.889967 738 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:03:12.957238 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38861
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:13.579845 752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:13.580199 752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:13.580442 752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:13.589984 752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:13.590199 752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:13.590476 752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:03:13.602813 752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38861
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:13.604315 752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:13.606048 752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:13.624392 758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:13.624902 762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:13.625114 757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:13.626508 752 server_base.cc:1034] running on GCE node
I20260809 12:03:13.627329 752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:13.628585 752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:13.629913 752 hybrid_clock.cc:648] HybridClock initialized: now 1786276993629762 us; error 69 us; skew 500 ppm
I20260809 12:03:13.630297 752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:13 dist-test-slave-sb2z krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:13.634517 752 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:03:13.636282 752 webserver.cc:460] Webserver started at http://127.0.147.65:40973/ using document root <none> and password file <none>
I20260809 12:03:13.637071 752 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:13.637221 752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:13.637537 752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:13.640895 752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "a6eaadabcd724836823e1cd444eeb783"
format_stamp: "Formatted at 2026-08-09 12:03:13 on dist-test-slave-sb2z"
server_key: "a53248ed50ce727aa2b5e8460e62a0e3"
server_key_iv: "444d3bc67f55d46608b2f87673481b4b"
server_key_version: "encryptionkey@0"
I20260809 12:03:13.641764 752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "a6eaadabcd724836823e1cd444eeb783"
format_stamp: "Formatted at 2026-08-09 12:03:13 on dist-test-slave-sb2z"
server_key: "a53248ed50ce727aa2b5e8460e62a0e3"
server_key_iv: "444d3bc67f55d46608b2f87673481b4b"
server_key_version: "encryptionkey@0"
I20260809 12:03:13.647804 752 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260809 12:03:13.652282 769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:13.653823 752 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 12:03:13.654111 752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "a6eaadabcd724836823e1cd444eeb783"
format_stamp: "Formatted at 2026-08-09 12:03:13 on dist-test-slave-sb2z"
server_key: "a53248ed50ce727aa2b5e8460e62a0e3"
server_key_iv: "444d3bc67f55d46608b2f87673481b4b"
server_key_version: "encryptionkey@0"
I20260809 12:03:13.654348 752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:13.672366 752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:13.677786 752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:13.678212 752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:13.679723 752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:13.681790 752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:13.681870 752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:13.682125 752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:13.682224 752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:13.765138 752 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:43469
I20260809 12:03:13.765259 917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:43469 every 8 connection(s)
I20260809 12:03:13.766671 752 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:03:13.772500 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 752
I20260809 12:03:13.772787 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:03:13.773543 589 external_mini_cluster.cc:1428] Setting key 8f1862c77ae45850889fc26c24488ac9
Aug 09 12:03:13 dist-test-slave-sb2z krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:13.794894 922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:13.770343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:47075 (local address 127.0.147.126:38861)
0809 12:03:13.771131 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:13.771136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:13.772326 (+ 1190us) server_negotiation.cc:407] Connection header received
0809 12:03:13.774854 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:13.774858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:13.774922 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:13.775017 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:13.777150 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:13.778077 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:13.780358 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:13.780934 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:13.781091 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:13.785880 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:13.785890 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:13.785898 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:13.785944 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:13.790600 (+ 4656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:13.791408 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:13.791417 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:13.791423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:13.791756 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:13.792391 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:13.792396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:13.792400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:13.792856 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:13.793037 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:13.793963 (+ 926us) server_negotiation.cc:299] Negotiation successful
0809 12:03:13.794278 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":497,"threads_started":1}
I20260809 12:03:13.795737 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:13.770951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38861 (local address 127.0.147.65:47075)
0809 12:03:13.772163 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:13.772208 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:13.774550 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:13.775350 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:13.775362 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:13.776207 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:13.776961 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:13.776981 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:13.778267 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:13.778272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:13.780202 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:13.780215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:13.781259 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:13.781270 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:13.781603 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:13.782358 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:13.782379 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:13.785573 (+ 3194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:13.790753 (+ 5180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:13.790760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:13.790772 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:13.791261 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:13.791916 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:13.791923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:13.791926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:13.792252 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:13.792979 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:13.792984 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:13.793436 (+ 452us) client_negotiation.cc:769] Sending connection context
0809 12:03:13.793746 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 12:03:13.794196 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"thread_start_us":560,"threads_started":1}
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
I20260809 12:03:13.798066 918 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38861
I20260809 12:03:13.798552 918 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:13.799769 918 heartbeater.cc:511] Master 127.0.147.126:38861 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
I20260809 12:03:13.804054 661 ts_manager.cc:194] Registered new tserver with Master: a6eaadabcd724836823e1cd444eeb783 (127.0.147.65:43469)
I20260809 12:03:13.807286 661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:47075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:03:13.829054 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38861
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:14.348054 927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:14.348420 927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:14.348655 927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:14.357774 927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:14.357981 927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:14.358192 927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:03:14.369839 927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38861
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:14.371255 927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:14.372763 927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:14.389919 934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:14.390537 935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:14.390889 937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:14.392447 927 server_base.cc:1034] running on GCE node
I20260809 12:03:14.393047 927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:14.393998 927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:14.395278 927 hybrid_clock.cc:648] HybridClock initialized: now 1786276994395191 us; error 48 us; skew 500 ppm
I20260809 12:03:14.395709 927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:14 dist-test-slave-sb2z krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276994, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:14.399857 927 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:03:14.401772 927 webserver.cc:460] Webserver started at http://127.0.147.66:40519/ using document root <none> and password file <none>
I20260809 12:03:14.402628 927 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:14.402814 927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:14.403199 927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:14.406572 927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "3bd63f2013654363844a7944109d8c34"
format_stamp: "Formatted at 2026-08-09 12:03:14 on dist-test-slave-sb2z"
server_key: "ffdd9e4a63a335ffe10619e89286e84c"
server_key_iv: "2ced8577cf9174ee1c859b433e06ef7c"
server_key_version: "encryptionkey@0"
I20260809 12:03:14.407640 927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "3bd63f2013654363844a7944109d8c34"
format_stamp: "Formatted at 2026-08-09 12:03:14 on dist-test-slave-sb2z"
server_key: "ffdd9e4a63a335ffe10619e89286e84c"
server_key_iv: "2ced8577cf9174ee1c859b433e06ef7c"
server_key_version: "encryptionkey@0"
I20260809 12:03:14.413816 927 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260809 12:03:14.418027 946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:14.419323 927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:03:14.419596 927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "3bd63f2013654363844a7944109d8c34"
format_stamp: "Formatted at 2026-08-09 12:03:14 on dist-test-slave-sb2z"
server_key: "ffdd9e4a63a335ffe10619e89286e84c"
server_key_iv: "2ced8577cf9174ee1c859b433e06ef7c"
server_key_version: "encryptionkey@0"
I20260809 12:03:14.419868 927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:14.437604 927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:14.442525 927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:14.442901 927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:14.444394 927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:14.446063 927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:14.446180 927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:14.446379 927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:14.446465 927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:14.516105 927 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:38559
I20260809 12:03:14.516209 1097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:38559 every 8 connection(s)
I20260809 12:03:14.517612 927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:03:14.522528 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 927
I20260809 12:03:14.522768 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:03:14.523391 589 external_mini_cluster.cc:1428] Setting key d5f7b46049891fd5cb2c33c2b8acc266
Aug 09 12:03:14 dist-test-slave-sb2z krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276994, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:14.549890 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:14.521243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:44569 (local address 127.0.147.126:38861)
0809 12:03:14.521932 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:14.521937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:14.523292 (+ 1355us) server_negotiation.cc:407] Connection header received
0809 12:03:14.526066 (+ 2774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:14.526072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:14.526158 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:14.526381 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:14.529233 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:14.530945 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:14.534208 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:14.535055 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:14.535165 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:14.540706 (+ 5541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:14.540716 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:14.540724 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:14.540772 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:14.545954 (+ 5182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:14.546786 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:14.546795 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:14.546802 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:14.547063 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:14.547605 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:14.547613 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:14.547618 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:14.548065 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:14.548226 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:14.549100 (+ 874us) server_negotiation.cc:299] Negotiation successful
0809 12:03:14.549409 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":412,"threads_started":1}
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
I20260809 12:03:14.550597 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:14.521793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38861 (local address 127.0.147.66:44569)
0809 12:03:14.523096 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:14.523144 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:14.525801 (+ 2657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:14.526717 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:14.526731 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:14.527914 (+ 1183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:14.528986 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:14.529025 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:14.531144 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:14.531153 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:14.534007 (+ 2854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:14.534026 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:14.535388 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:14.535399 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:14.535863 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:14.536806 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:14.536831 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:14.540418 (+ 3587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:14.546127 (+ 5709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:14.546135 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:14.546149 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:14.546636 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:14.547181 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:14.547184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:14.547186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:14.547456 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:14.548208 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:14.548214 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:14.548619 (+ 405us) client_negotiation.cc:769] Sending connection context
0809 12:03:14.548894 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 12:03:14.549322 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"thread_start_us":670,"threads_started":1}
I20260809 12:03:14.552251 1098 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38861
I20260809 12:03:14.552654 1098 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
I20260809 12:03:14.553613 1098 heartbeater.cc:511] Master 127.0.147.126:38861 requested a full tablet report, sending...
I20260809 12:03:14.555467 661 ts_manager.cc:194] Registered new tserver with Master: 3bd63f2013654363844a7944109d8c34 (127.0.147.66:38559)
I20260809 12:03:14.556348 661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:44569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:03:14.581123 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38861
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:03:14.811769 918 heartbeater.cc:503] Master 127.0.147.126:38861 was elected leader, sending a full tablet report...
W20260809 12:03:15.120097 1108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:15.120428 1108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:15.120587 1108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:15.129664 1108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:15.129841 1108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:15.130049 1108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:03:15.141957 1108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38861
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:15.143373 1108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:15.144927 1108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:15.164100 1113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:15.165278 1120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:15.165242 1116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:15.166601 1108 server_base.cc:1034] running on GCE node
I20260809 12:03:15.167363 1108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:15.168387 1108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:15.169621 1108 hybrid_clock.cc:648] HybridClock initialized: now 1786276995169559 us; error 57 us; skew 500 ppm
I20260809 12:03:15.169972 1108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:15 dist-test-slave-sb2z krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:15.174101 1108 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:03:15.176159 1108 webserver.cc:460] Webserver started at http://127.0.147.67:40185/ using document root <none> and password file <none>
I20260809 12:03:15.176918 1108 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:15.177114 1108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:15.177541 1108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:15.181468 1108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "6b3cdeefdf464fad957655e5f84c0c98"
format_stamp: "Formatted at 2026-08-09 12:03:15 on dist-test-slave-sb2z"
server_key: "8066ce150037b37112bba82fdf5f40bc"
server_key_iv: "b4ded45b61fa7d05db2595b0515c3a61"
server_key_version: "encryptionkey@0"
I20260809 12:03:15.182461 1108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "6b3cdeefdf464fad957655e5f84c0c98"
format_stamp: "Formatted at 2026-08-09 12:03:15 on dist-test-slave-sb2z"
server_key: "8066ce150037b37112bba82fdf5f40bc"
server_key_iv: "b4ded45b61fa7d05db2595b0515c3a61"
server_key_version: "encryptionkey@0"
I20260809 12:03:15.191378 1108 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s
I20260809 12:03:15.196286 1130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:15.197935 1108 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 12:03:15.198151 1108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "6b3cdeefdf464fad957655e5f84c0c98"
format_stamp: "Formatted at 2026-08-09 12:03:15 on dist-test-slave-sb2z"
server_key: "8066ce150037b37112bba82fdf5f40bc"
server_key_iv: "b4ded45b61fa7d05db2595b0515c3a61"
server_key_version: "encryptionkey@0"
I20260809 12:03:15.198490 1108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:15.222307 1108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:15.227809 1108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:15.228269 1108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:15.229830 1108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:15.231943 1108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:15.232122 1108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:15.232345 1108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:15.232493 1108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:15.326057 1108 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:38713
I20260809 12:03:15.326051 1281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:38713 every 8 connection(s)
I20260809 12:03:15.328594 1108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:03:15.336786 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 1108
I20260809 12:03:15.337086 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:03:15.337832 589 external_mini_cluster.cc:1428] Setting key aa4ce43f2a1d995b38918205f5756a96
Aug 09 12:03:15 dist-test-slave-sb2z krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:15.364498 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:15.339779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:34337 (local address 127.0.147.126:38861)
0809 12:03:15.340438 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:15.340443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:15.340462 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:03:15.342772 (+ 2310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:15.342778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:15.342906 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:15.343052 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:15.345703 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.347208 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.350539 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.351063 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.351166 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:15.357880 (+ 6714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:15.357885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:15.357888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:15.357937 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:15.361110 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.361903 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.361909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.361913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.362049 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.362574 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.362579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.362582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.362874 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:15.363032 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:15.363864 (+ 832us) server_negotiation.cc:299] Negotiation successful
0809 12:03:15.364087 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":378,"threads_started":1}
I20260809 12:03:15.365828 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:15.334522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38861 (local address 127.0.147.67:34337)
0809 12:03:15.336058 (+ 1536us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:15.336105 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:15.342459 (+ 6354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:15.343314 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:15.343344 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:15.344352 (+ 1008us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:15.345495 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:15.345526 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.347437 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.347445 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:15.350385 (+ 2940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:15.350403 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.351408 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.351419 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:15.351966 (+ 547us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:15.352957 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:15.352995 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:15.357570 (+ 4575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:15.361273 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:15.361281 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:15.361296 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:15.361756 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:15.362167 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:15.362172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:15.362176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:15.362446 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:15.363011 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:15.363018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:15.363424 (+ 406us) client_negotiation.cc:769] Sending connection context
0809 12:03:15.363764 (+ 340us) client_negotiation.cc:240] Negotiation successful
0809 12:03:15.364318 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1153,"thread_start_us":707,"threads_started":1}
I20260809 12:03:15.367466 1282 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38861
I20260809 12:03:15.367859 1282 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:15.369114 1282 heartbeater.cc:511] Master 127.0.147.126:38861 requested a full tablet report, sending...
I20260809 12:03:15.370738 661 ts_manager.cc:194] Registered new tserver with Master: 6b3cdeefdf464fad957655e5f84c0c98 (127.0.147.67:38713)
I20260809 12:03:15.371876 661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:34337
I20260809 12:03:15.379752 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:03:15 dist-test-slave-sb2z krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276995, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:03:15 dist-test-slave-sb2z krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276995, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:15.440577 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:15.419432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.0.147.65:43469)
0809 12:03:15.420200 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:15.420220 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:15.421152 (+ 932us) server_negotiation.cc:407] Connection header received
0809 12:03:15.421357 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:15.421370 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:15.421589 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:15.421772 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:15.424161 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.425253 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.427048 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.427712 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.427858 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:15.433736 (+ 5878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:15.433747 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:15.433762 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:15.433801 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:15.436775 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.437596 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.437602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.437605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.437732 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.438191 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.438194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.438196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.438625 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:15.438891 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:15.439250 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 12:03:15.439486 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":363,"threads_started":1}
W20260809 12:03:15.442706 802 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39454
Aug 09 12:03:15 dist-test-slave-sb2z krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:03:15 dist-test-slave-sb2z krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:15.501480 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:15.481568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39460 (local address 127.0.147.65:43469)
0809 12:03:15.481791 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:15.481796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:15.481895 (+ 99us) server_negotiation.cc:407] Connection header received
0809 12:03:15.482049 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:15.482053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:15.482128 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:15.482213 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:15.483579 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.484514 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.485696 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.486407 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.486502 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:15.494280 (+ 7778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:15.494285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:15.494288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:15.494324 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:15.498346 (+ 4022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.499290 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.499294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.499296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.499504 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.500006 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.500010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.500011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.500303 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:15.500559 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:15.500806 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 12:03:15.501016 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:03:15.503470 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 752
I20260809 12:03:15.521636 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:15.505091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.0.147.65:43469)
0809 12:03:15.505269 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:15.505273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:15.505809 (+ 536us) server_negotiation.cc:407] Connection header received
0809 12:03:15.505944 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:15.505948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:15.506017 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:15.506101 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:15.507956 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.508665 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.510271 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.510918 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.511014 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:15.513531 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:15.513536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:15.513539 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:15.513568 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:15.518272 (+ 4704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.519042 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.519049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.519052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.519361 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.519872 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.519886 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.519889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.520320 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:15.520610 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:15.520788 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 12:03:15.521042 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:03:15.522879 912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:15.560885 1098 heartbeater.cc:503] Master 127.0.147.126:38861 was elected leader, sending a full tablet report...
I20260809 12:03:15.612514 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 752
I20260809 12:03:15.639650 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 927
Aug 09 12:03:15 dist-test-slave-sb2z krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:03:15.656183 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:15.640836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37112 (local address 127.0.147.66:38559)
0809 12:03:15.641477 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:15.641485 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:15.641510 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:03:15.641582 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:15.641588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:15.641788 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:15.641910 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:15.643066 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.644212 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.645392 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.645941 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.646043 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:15.650464 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:15.650475 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:15.650488 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:15.650523 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:15.653458 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.654230 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.654237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.654239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.654353 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.654790 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.654793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.654795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.655109 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:15.655335 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:15.655702 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 12:03:15.655895 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":326,"threads_started":1}
I20260809 12:03:15.657538 1091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:15.756191 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 927
I20260809 12:03:15.781565 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 1108
Aug 09 12:03:15 dist-test-slave-sb2z krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:03:15.797899 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:15.782627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35050 (local address 127.0.147.67:38713)
0809 12:03:15.783354 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:15.783364 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:15.783405 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:03:15.783486 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:15.783502 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:15.783791 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:15.784102 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:15.785458 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.786454 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.787844 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:15.788385 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:15.788485 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:15.792539 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:15.792549 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:15.792564 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:15.792600 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:15.795277 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.795936 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.795943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.795945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.796073 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:15.796418 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:15.796421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:15.796423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:15.796807 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:15.797038 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:15.797290 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 12:03:15.797511 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":341,"threads_started":1}
I20260809 12:03:15.799183 1276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:15.883338 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 1108
I20260809 12:03:15.909274 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 619
I20260809 12:03:15.910504 705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:16.016160 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 619
2026-08-09T12:03:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5685 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:03:16 dist-test-slave-sb2z krb5kdc[1318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:03:16 dist-test-slave-sb2z krb5kdc[1318](info): set up 2 sockets
Aug 09 12:03:16 dist-test-slave-sb2z krb5kdc[1318](info): commencing operation
krb5kdc: starting...
W20260809 12:03:17.335001 589 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:03:17 dist-test-slave-sb2z krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:03:17.533179 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39723
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:39723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:17.861320 1336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:17.861610 1336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:17.861702 1336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:17.868847 1336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:17.868952 1336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:17.868999 1336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:17.869060 1336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:17.869105 1336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:17.878579 1336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:39723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:17.879824 1336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:17.881058 1336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:17.892634 1347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:17.892745 1349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:17.892786 1345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:17.893713 1336 server_base.cc:1034] running on GCE node
I20260809 12:03:17.894425 1336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:17.895808 1336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:17.897002 1336 hybrid_clock.cc:648] HybridClock initialized: now 1786276997896951 us; error 44 us; skew 500 ppm
I20260809 12:03:17.897344 1336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:17 dist-test-slave-sb2z krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276997, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:17.900645 1336 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:03:17.902050 1336 webserver.cc:460] Webserver started at http://127.0.147.126:35281/ using document root <none> and password file <none>
I20260809 12:03:17.902732 1336 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:17.902861 1336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:17.903191 1336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:17.906250 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "387410ce33b74870aa469e5937798da3"
format_stamp: "Formatted at 2026-08-09 12:03:17 on dist-test-slave-sb2z"
server_key: "876d6a0b814ebd51f727dc4f7f2b44e1"
server_key_iv: "1f61c5360cc7e4f0ffba4a2931ff3d58"
server_key_version: "encryptionkey@0"
I20260809 12:03:17.907013 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "387410ce33b74870aa469e5937798da3"
format_stamp: "Formatted at 2026-08-09 12:03:17 on dist-test-slave-sb2z"
server_key: "876d6a0b814ebd51f727dc4f7f2b44e1"
server_key_iv: "1f61c5360cc7e4f0ffba4a2931ff3d58"
server_key_version: "encryptionkey@0"
I20260809 12:03:17.912257 1336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:03:17.915477 1356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:17.916735 1336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:03:17.916913 1336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "387410ce33b74870aa469e5937798da3"
format_stamp: "Formatted at 2026-08-09 12:03:17 on dist-test-slave-sb2z"
server_key: "876d6a0b814ebd51f727dc4f7f2b44e1"
server_key_iv: "1f61c5360cc7e4f0ffba4a2931ff3d58"
server_key_version: "encryptionkey@0"
I20260809 12:03:17.917181 1336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:17.933790 1336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:17.937937 1336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:17.938237 1336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:17.956807 1336 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:39723
I20260809 12:03:17.956846 1423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:39723 every 8 connection(s)
I20260809 12:03:17.958130 1336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:17.961936 1424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:17.961939 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 1336
I20260809 12:03:17.962184 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:17.963039 589 external_mini_cluster.cc:1428] Setting key ad474021ab64977bdd0df66555016ecb
I20260809 12:03:17.976055 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3: Bootstrap starting.
Aug 09 12:03:17 dist-test-slave-sb2z krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:17.979645 1424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:17.980868 1424 log.cc:824] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:17.983722 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3: No bootstrap required, opened a new log
I20260809 12:03:17.986526 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:17.965137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.0.147.126:39723)
0809 12:03:17.965971 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:17.965982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:17.966028 (+ 46us) server_negotiation.cc:407] Connection header received
0809 12:03:17.968260 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:17.968281 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:17.968655 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:17.969034 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:17.970808 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:17.972129 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:17.973933 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:17.974555 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:17.974677 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:17.979265 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:17.979272 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:17.979285 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:17.979318 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:17.982453 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:17.983161 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:17.983168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:17.983173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:17.983351 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:17.983798 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:17.983804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:17.983809 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:17.984335 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:17.984664 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:17.985161 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 12:03:17.985577 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":433,"threads_started":1}
I20260809 12:03:17.991899 1424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387410ce33b74870aa469e5937798da3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39723 } }
I20260809 12:03:17.992554 1424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:17.992697 1424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387410ce33b74870aa469e5937798da3, State: Initialized, Role: FOLLOWER
I20260809 12:03:17.993409 1424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387410ce33b74870aa469e5937798da3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39723 } }
I20260809 12:03:17.993697 1424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:17.993865 1424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:17.994078 1424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:17.996536 1424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387410ce33b74870aa469e5937798da3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39723 } }
I20260809 12:03:17.997081 1424 leader_election.cc:302] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 387410ce33b74870aa469e5937798da3; no voters:
I20260809 12:03:17.997711 1424 leader_election.cc:288] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:17.997829 1432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:17.999457 1432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [term 1 LEADER]: Becoming Leader. State: Replica: 387410ce33b74870aa469e5937798da3, State: Running, Role: LEADER
I20260809 12:03:17.999969 1432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387410ce33b74870aa469e5937798da3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39723 } }
I20260809 12:03:18.000238 1424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:18.002964 1433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "387410ce33b74870aa469e5937798da3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387410ce33b74870aa469e5937798da3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39723 } } }
I20260809 12:03:18.002928 1434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 387410ce33b74870aa469e5937798da3. Latest consensus state: current_term: 1 leader_uuid: "387410ce33b74870aa469e5937798da3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387410ce33b74870aa469e5937798da3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39723 } } }
I20260809 12:03:18.003453 1433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:18.003459 1434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:18.004316 1439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:18.009675 1439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:18.020454 1439 catalog_manager.cc:1347] Generated new cluster ID: 93fffcd790f54515a966ea5bb3bd6325
I20260809 12:03:18.020596 1439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:18.048997 1439 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:18.050530 1439 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:18.064247 1439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 387410ce33b74870aa469e5937798da3: Generated new TSK 0
I20260809 12:03:18.065181 1439 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:03:18.122320 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39723
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:18.455400 1460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:18.455705 1460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:18.455894 1460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:18.463181 1460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:18.463282 1460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:18.463486 1460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:03:18.473140 1460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39723
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:18.474372 1460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:18.475716 1460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:18.489219 1468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:18.489324 1467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:18.489348 1472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:18.490271 1460 server_base.cc:1034] running on GCE node
I20260809 12:03:18.490902 1460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:18.491928 1460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:18.493165 1460 hybrid_clock.cc:648] HybridClock initialized: now 1786276998493123 us; error 54 us; skew 500 ppm
I20260809 12:03:18.493584 1460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:18 dist-test-slave-sb2z krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:18.496824 1460 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:03:18.498291 1460 webserver.cc:460] Webserver started at http://127.0.147.65:40211/ using document root <none> and password file <none>
I20260809 12:03:18.498994 1460 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:18.499166 1460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:18.499477 1460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:18.502501 1460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "a15f86377dea492bbe06854dae3958b9"
format_stamp: "Formatted at 2026-08-09 12:03:18 on dist-test-slave-sb2z"
server_key: "eb338ca8a6e086b5529fef9d0d343075"
server_key_iv: "87b6bb27c317db667ea7cc9a63cef9e1"
server_key_version: "encryptionkey@0"
I20260809 12:03:18.503335 1460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "a15f86377dea492bbe06854dae3958b9"
format_stamp: "Formatted at 2026-08-09 12:03:18 on dist-test-slave-sb2z"
server_key: "eb338ca8a6e086b5529fef9d0d343075"
server_key_iv: "87b6bb27c317db667ea7cc9a63cef9e1"
server_key_version: "encryptionkey@0"
I20260809 12:03:18.508507 1460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:03:18.511857 1482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:18.513094 1460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:03:18.513274 1460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "a15f86377dea492bbe06854dae3958b9"
format_stamp: "Formatted at 2026-08-09 12:03:18 on dist-test-slave-sb2z"
server_key: "eb338ca8a6e086b5529fef9d0d343075"
server_key_iv: "87b6bb27c317db667ea7cc9a63cef9e1"
server_key_version: "encryptionkey@0"
I20260809 12:03:18.513486 1460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:18.539093 1460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:18.543689 1460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:18.544010 1460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:18.545018 1460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:18.546388 1460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:18.546499 1460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:18.546656 1460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:18.546732 1460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:18.580945 1460 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:36743
I20260809 12:03:18.581040 1634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:36743 every 8 connection(s)
I20260809 12:03:18.582350 1460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:03:18.591851 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 1460
I20260809 12:03:18.592083 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:03:18.592835 589 external_mini_cluster.cc:1428] Setting key c119a6828ccaac9f78b5c5b7271e1a5f
Aug 09 12:03:18 dist-test-slave-sb2z krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:18.602906 1639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:18.584862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:53281 (local address 127.0.147.126:39723)
0809 12:03:18.585320 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:18.585325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:18.586130 (+ 805us) server_negotiation.cc:407] Connection header received
0809 12:03:18.588074 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:18.588081 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:18.588182 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:18.588304 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:18.589766 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:18.590503 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:18.592540 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:18.593004 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:18.593114 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:18.596924 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:18.596933 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:18.596941 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:18.596985 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:18.599601 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:18.600388 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:18.600395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:18.600405 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:18.600533 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:18.601029 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:18.601034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:18.601044 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:18.601368 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:18.601538 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:18.602301 (+ 763us) server_negotiation.cc:299] Negotiation successful
0809 12:03:18.602538 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":255,"threads_started":1}
I20260809 12:03:18.603929 1640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:18.585271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39723 (local address 127.0.147.65:53281)
0809 12:03:18.585955 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:18.586002 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:18.587799 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:18.588534 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:18.588542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:18.589040 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:18.589593 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:18.589609 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:18.590676 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:18.590680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:18.592380 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:18.592403 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:18.593244 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:18.593252 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:18.593536 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:18.594222 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:18.594243 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:18.596659 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:18.599733 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:18.599742 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:18.599756 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:18.600260 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:18.600647 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:18.600652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:18.600656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:18.600894 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:18.601501 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:18.601509 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:18.601901 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 12:03:18.602186 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 12:03:18.602702 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":270,"threads_started":1}
I20260809 12:03:18.605554 1636 heartbeater.cc:347] Connected to a master server at 127.0.147.126:39723
I20260809 12:03:18.605852 1636 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:18.606531 1636 heartbeater.cc:511] Master 127.0.147.126:39723 requested a full tablet report, sending...
I20260809 12:03:18.608693 1378 ts_manager.cc:194] Registered new tserver with Master: a15f86377dea492bbe06854dae3958b9 (127.0.147.65:36743)
I20260809 12:03:18.610245 1378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:53281
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:03:18.640314 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39723
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:18.977056 1644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:18.977372 1644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:18.977545 1644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:18.984930 1644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:18.985098 1644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:18.985284 1644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:03:18.995009 1644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39723
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:18.996328 1644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:18.997648 1644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:19.011229 1659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:19.011195 1656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:19.011593 1655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:19.012006 1644 server_base.cc:1034] running on GCE node
I20260809 12:03:19.012642 1644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:19.013533 1644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:19.014755 1644 hybrid_clock.cc:648] HybridClock initialized: now 1786276999014685 us; error 55 us; skew 500 ppm
I20260809 12:03:19.015100 1644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:19 dist-test-slave-sb2z krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:19.018666 1644 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:03:19.020375 1644 webserver.cc:460] Webserver started at http://127.0.147.66:42633/ using document root <none> and password file <none>
I20260809 12:03:19.021159 1644 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:19.021311 1644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:19.021634 1644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:19.025051 1644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "7f1585337d6246deb49f6eb2c6650851"
format_stamp: "Formatted at 2026-08-09 12:03:19 on dist-test-slave-sb2z"
server_key: "0f37cdb9fded6440cc78548cd26a1d79"
server_key_iv: "eab0cd1fc6cddc0deebdbec72c781a7d"
server_key_version: "encryptionkey@0"
I20260809 12:03:19.025887 1644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "7f1585337d6246deb49f6eb2c6650851"
format_stamp: "Formatted at 2026-08-09 12:03:19 on dist-test-slave-sb2z"
server_key: "0f37cdb9fded6440cc78548cd26a1d79"
server_key_iv: "eab0cd1fc6cddc0deebdbec72c781a7d"
server_key_version: "encryptionkey@0"
I20260809 12:03:19.031308 1644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:03:19.034951 1667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:19.036398 1644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:19.036598 1644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "7f1585337d6246deb49f6eb2c6650851"
format_stamp: "Formatted at 2026-08-09 12:03:19 on dist-test-slave-sb2z"
server_key: "0f37cdb9fded6440cc78548cd26a1d79"
server_key_iv: "eab0cd1fc6cddc0deebdbec72c781a7d"
server_key_version: "encryptionkey@0"
I20260809 12:03:19.036828 1644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:19.056424 1644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:19.061060 1644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:19.061378 1644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:19.062388 1644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:19.063876 1644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:19.063984 1644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:19.064150 1644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:19.064229 1644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:19.100179 1644 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:35161
I20260809 12:03:19.100270 1815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:35161 every 8 connection(s)
I20260809 12:03:19.101765 1644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:03:19.110273 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 1644
I20260809 12:03:19.110492 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:03:19.110926 589 external_mini_cluster.cc:1428] Setting key 251de793d7c74e6ae6527ea6f8403753
Aug 09 12:03:19 dist-test-slave-sb2z krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:19.125262 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.104632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:52759 (local address 127.0.147.126:39723)
0809 12:03:19.105061 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:19.105067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:19.106131 (+ 1064us) server_negotiation.cc:407] Connection header received
0809 12:03:19.108125 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:19.108131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:19.108231 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:19.108362 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:19.109877 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.110720 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.112689 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.113396 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.113534 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.117834 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:19.117843 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:19.117851 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:19.117899 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:19.121830 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:19.122818 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:19.122822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:19.122824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:19.122948 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:19.123445 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:19.123448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:19.123450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:19.123785 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:19.123910 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:19.124818 (+ 908us) server_negotiation.cc:299] Negotiation successful
0809 12:03:19.124989 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":212,"threads_started":1}
I20260809 12:03:19.126883 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.105225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39723 (local address 127.0.147.66:52759)
0809 12:03:19.105955 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:19.106004 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:19.107815 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:19.108607 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:19.108615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:19.109140 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:19.109695 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.109712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.110869 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.110873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.112541 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.112556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.113617 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.113623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.113832 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.114589 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:19.114617 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:19.117533 (+ 2916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:19.121974 (+ 4441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:19.121986 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:19.122005 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:19.122687 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:19.123061 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:19.123066 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:19.123070 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:19.123331 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:19.123909 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:19.123920 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:19.124386 (+ 466us) client_negotiation.cc:769] Sending connection context
0809 12:03:19.124732 (+ 346us) client_negotiation.cc:240] Negotiation successful
0809 12:03:19.125335 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":291,"threads_started":1}
I20260809 12:03:19.128461 1819 heartbeater.cc:347] Connected to a master server at 127.0.147.126:39723
I20260809 12:03:19.128772 1819 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:19.129514 1819 heartbeater.cc:511] Master 127.0.147.126:39723 requested a full tablet report, sending...
I20260809 12:03:19.131019 1378 ts_manager.cc:194] Registered new tserver with Master: 7f1585337d6246deb49f6eb2c6650851 (127.0.147.66:35161)
I20260809 12:03:19.132493 1378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:52759
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:03:19.162281 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39723
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:19.498234 1830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:19.498503 1830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:19.498683 1830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:19.506161 1830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:19.506299 1830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:19.506488 1830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:03:19.516410 1830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39723
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:19.517850 1830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:19.519187 1830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:19.532361 1837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:19.532425 1836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:19.532449 1839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:19.533615 1830 server_base.cc:1034] running on GCE node
I20260809 12:03:19.534240 1830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:19.535192 1830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:19.536386 1830 hybrid_clock.cc:648] HybridClock initialized: now 1786276999536328 us; error 54 us; skew 500 ppm
I20260809 12:03:19.536706 1830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:19 dist-test-slave-sb2z krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:19.539927 1830 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:03:19.541504 1830 webserver.cc:460] Webserver started at http://127.0.147.67:44253/ using document root <none> and password file <none>
I20260809 12:03:19.542202 1830 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:19.542352 1830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:19.542673 1830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:19.545634 1830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "1ac4457ebbf4473fb533dc57b079af41"
format_stamp: "Formatted at 2026-08-09 12:03:19 on dist-test-slave-sb2z"
server_key: "7858ec1112846a63395b5229d6a50c9e"
server_key_iv: "bad35a922061e9c1a17a7e781cf3007f"
server_key_version: "encryptionkey@0"
I20260809 12:03:19.546451 1830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "1ac4457ebbf4473fb533dc57b079af41"
format_stamp: "Formatted at 2026-08-09 12:03:19 on dist-test-slave-sb2z"
server_key: "7858ec1112846a63395b5229d6a50c9e"
server_key_iv: "bad35a922061e9c1a17a7e781cf3007f"
server_key_version: "encryptionkey@0"
I20260809 12:03:19.551908 1830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:03:19.555416 1851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:19.556805 1830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:19.556991 1830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "1ac4457ebbf4473fb533dc57b079af41"
format_stamp: "Formatted at 2026-08-09 12:03:19 on dist-test-slave-sb2z"
server_key: "7858ec1112846a63395b5229d6a50c9e"
server_key_iv: "bad35a922061e9c1a17a7e781cf3007f"
server_key_version: "encryptionkey@0"
I20260809 12:03:19.557219 1830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:19.576583 1830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:19.581172 1830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:19.581473 1830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:19.582499 1830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:19.583989 1830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:19.584115 1830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:19.584277 1830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:19.584349 1830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:19.613226 1636 heartbeater.cc:503] Master 127.0.147.126:39723 was elected leader, sending a full tablet report...
I20260809 12:03:19.624318 1830 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:45031
I20260809 12:03:19.624428 1999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:45031 every 8 connection(s)
I20260809 12:03:19.625818 1830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:03:19.632205 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 1830
I20260809 12:03:19.632411 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:03:19.633117 589 external_mini_cluster.cc:1428] Setting key 5272c63b38ae404913717803fc8f26b4
Aug 09 12:03:19 dist-test-slave-sb2z krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:19.653051 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.629302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:33975 (local address 127.0.147.126:39723)
0809 12:03:19.629982 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:19.629989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:19.631451 (+ 1462us) server_negotiation.cc:407] Connection header received
0809 12:03:19.635160 (+ 3709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:19.635165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:19.635314 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:19.635409 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:19.637543 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.638706 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.640821 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.641377 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.641475 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.645843 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:19.645848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:19.645851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:19.645882 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:19.649982 (+ 4100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:19.650691 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:19.650695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:19.650697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:19.650856 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:19.651327 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:19.651331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:19.651332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:19.651651 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:19.651774 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:19.652484 (+ 710us) server_negotiation.cc:299] Negotiation successful
0809 12:03:19.652659 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":329,"threads_started":1}
I20260809 12:03:19.654014 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.630248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39723 (local address 127.0.147.67:33975)
0809 12:03:19.631257 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:19.631328 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:19.634805 (+ 3477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:19.635701 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:19.635709 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:19.636606 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:19.637388 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.637407 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.638847 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.638851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.640693 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.640706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.641767 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.641778 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.642004 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.642711 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:19.642731 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:19.645631 (+ 2900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:19.650120 (+ 4489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:19.650127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:19.650138 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:19.650579 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:19.650972 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:19.650976 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:19.650979 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:19.651221 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:19.651766 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:19.651771 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:19.652084 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 12:03:19.652355 (+ 271us) client_negotiation.cc:240] Negotiation successful
0809 12:03:19.652763 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":514,"threads_started":1}
I20260809 12:03:19.655352 2000 heartbeater.cc:347] Connected to a master server at 127.0.147.126:39723
I20260809 12:03:19.655776 2000 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:19.656808 2000 heartbeater.cc:511] Master 127.0.147.126:39723 requested a full tablet report, sending...
I20260809 12:03:19.658512 1378 ts_manager.cc:194] Registered new tserver with Master: 1ac4457ebbf4473fb533dc57b079af41 (127.0.147.67:45031)
I20260809 12:03:19.659626 1378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:33975
I20260809 12:03:19.661320 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:03:19.691175 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.669665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60686 (local address 127.0.147.126:39723)
0809 12:03:19.669817 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:19.669821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:19.670817 (+ 996us) server_negotiation.cc:407] Connection header received
0809 12:03:19.671034 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:19.671038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:19.671103 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:19.671182 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:19.674576 (+ 3394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.675508 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.678202 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.678760 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.678866 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.683440 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:19.683445 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:19.683449 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:19.683481 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:19.687596 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:19.688572 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:19.688576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:19.688579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:19.688757 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:19.689248 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:19.689251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:19.689254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:19.689658 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:19.689920 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:19.690519 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 12:03:19.690782 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:03:19.702335 1378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:03:19.713416 1378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:19.755326 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.733898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.0.147.67:45031)
0809 12:03:19.737319 (+ 3421us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:19.737332 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:19.741758 (+ 4426us) server_negotiation.cc:407] Connection header received
0809 12:03:19.742103 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:19.742112 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:19.742407 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:19.742707 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:19.744406 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.747203 (+ 2797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.751027 (+ 3824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.752954 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.753243 (+ 289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.753635 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:19.754147 (+ 512us) server_negotiation.cc:299] Negotiation successful
0809 12:03:19.754380 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3105,"thread_start_us":2866,"threads_started":1}
I20260809 12:03:19.755862 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.733742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45031 (local address 127.0.0.1:49270)
0809 12:03:19.741603 (+ 7861us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:19.741632 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:19.741855 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:19.742938 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:19.742955 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:19.743000 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:19.744049 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.744066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.747379 (+ 3313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.747386 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.750829 (+ 3443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.750842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.753270 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.753278 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.753805 (+ 527us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.753880 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 12:03:19.754012 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 12:03:19.754248 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7618,"mutex_wait_us":50,"thread_start_us":7528,"threads_started":1}
I20260809 12:03:19.757124 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.732904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:35161 (local address 127.0.0.1:60322)
0809 12:03:19.734601 (+ 1697us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:19.734640 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:19.734949 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:19.744194 (+ 9245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:19.744199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:19.744222 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:19.745357 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.745370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.747460 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.747465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.752105 (+ 4640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.752133 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.755291 (+ 3158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.755300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.756305 (+ 1005us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.756355 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 12:03:19.756454 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 12:03:19.756584 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1369,"mutex_wait_us":267,"thread_start_us":1131,"threads_started":1}
I20260809 12:03:19.758330 2022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.733419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.0.147.66:35161)
0809 12:03:19.743050 (+ 9631us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:19.743060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:19.743095 (+ 35us) server_negotiation.cc:407] Connection header received
0809 12:03:19.743259 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:19.743270 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:19.743931 (+ 661us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:19.745578 (+ 1647us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:19.745854 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.747273 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.752372 (+ 5099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.754782 (+ 2410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.755753 (+ 971us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.756079 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:19.757490 (+ 1411us) server_negotiation.cc:299] Negotiation successful
0809 12:03:19.757718 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9409,"thread_start_us":9136,"threads_started":1}
I20260809 12:03:19.759063 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.740927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.0.147.65:36743)
0809 12:03:19.744077 (+ 3150us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:19.744087 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:19.744132 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:03:19.744260 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:19.744270 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:19.745175 (+ 905us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:19.745533 (+ 358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:19.747127 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.749781 (+ 2654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.754132 (+ 4351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.757154 (+ 3022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.757347 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.757709 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:19.758256 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 12:03:19.758429 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2824,"thread_start_us":2453,"threads_started":1}
I20260809 12:03:19.759200 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.739758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36743 (local address 127.0.0.1:36476)
0809 12:03:19.741921 (+ 2163us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:19.741942 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:19.742118 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:19.745920 (+ 3802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:19.745926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:19.745951 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:19.746788 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.746805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.750014 (+ 3209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.750019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.753805 (+ 3786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.753822 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.757914 (+ 4092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.757927 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.758071 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.758093 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 12:03:19.758207 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 12:03:19.758337 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1954,"mutex_wait_us":32,"thread_start_us":1277,"threads_started":1}
I20260809 12:03:19.762272 1907 tablet_service.cc:1467] Processing CreateTablet for tablet 02631e23ab014acbb9d92999b52660a1 (DEFAULT_TABLE table=test-table [id=a575ae8390ac414e8d3f089b85f40d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:19.762688 1737 tablet_service.cc:1467] Processing CreateTablet for tablet 02631e23ab014acbb9d92999b52660a1 (DEFAULT_TABLE table=test-table [id=a575ae8390ac414e8d3f089b85f40d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:19.764505 1737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02631e23ab014acbb9d92999b52660a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:19.764710 1907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02631e23ab014acbb9d92999b52660a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:19.764178 1552 tablet_service.cc:1467] Processing CreateTablet for tablet 02631e23ab014acbb9d92999b52660a1 (DEFAULT_TABLE table=test-table [id=a575ae8390ac414e8d3f089b85f40d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:19.766216 1552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02631e23ab014acbb9d92999b52660a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:19.783179 2028 tablet_bootstrap.cc:492] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851: Bootstrap starting.
I20260809 12:03:19.787659 2029 tablet_bootstrap.cc:492] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41: Bootstrap starting.
I20260809 12:03:19.790369 2028 tablet_bootstrap.cc:654] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:19.791983 2030 tablet_bootstrap.cc:492] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9: Bootstrap starting.
I20260809 12:03:19.793299 2028 log.cc:824] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:19.794607 2029 tablet_bootstrap.cc:654] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:19.797369 2029 log.cc:824] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:19.798714 2030 tablet_bootstrap.cc:654] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:19.799691 2028 tablet_bootstrap.cc:492] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851: No bootstrap required, opened a new log
I20260809 12:03:19.800235 2028 ts_tablet_manager.cc:1397] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.000s
I20260809 12:03:19.801419 2030 log.cc:824] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:19.802553 2029 tablet_bootstrap.cc:492] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41: No bootstrap required, opened a new log
I20260809 12:03:19.803038 2029 ts_tablet_manager.cc:1397] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.013s
I20260809 12:03:19.805119 2030 tablet_bootstrap.cc:492] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9: No bootstrap required, opened a new log
I20260809 12:03:19.805480 2030 ts_tablet_manager.cc:1397] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260809 12:03:19.817623 2030 raft_consensus.cc:348] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } }
I20260809 12:03:19.818210 2030 raft_consensus.cc:374] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:19.818392 2030 raft_consensus.cc:729] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a15f86377dea492bbe06854dae3958b9, State: Initialized, Role: FOLLOWER
I20260809 12:03:19.819211 2030 consensus_queue.cc:260] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } }
I20260809 12:03:19.819640 2028 raft_consensus.cc:348] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } }
I20260809 12:03:19.820190 2028 raft_consensus.cc:374] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:19.820334 2028 raft_consensus.cc:729] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f1585337d6246deb49f6eb2c6650851, State: Initialized, Role: FOLLOWER
I20260809 12:03:19.821002 2029 raft_consensus.cc:348] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } }
I20260809 12:03:19.821689 2029 raft_consensus.cc:374] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:19.821502 2028 consensus_queue.cc:260] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } }
I20260809 12:03:19.821868 2029 raft_consensus.cc:729] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac4457ebbf4473fb533dc57b079af41, State: Initialized, Role: FOLLOWER
I20260809 12:03:19.822479 2030 ts_tablet_manager.cc:1428] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260809 12:03:19.823064 2029 consensus_queue.cc:260] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } }
I20260809 12:03:19.824572 1819 heartbeater.cc:503] Master 127.0.147.126:39723 was elected leader, sending a full tablet report...
I20260809 12:03:19.825412 2028 ts_tablet_manager.cc:1428] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851: Time spent starting tablet: real 0.025s user 0.012s sys 0.012s
I20260809 12:03:19.826790 2000 heartbeater.cc:503] Master 127.0.147.126:39723 was elected leader, sending a full tablet report...
I20260809 12:03:19.827366 2029 ts_tablet_manager.cc:1428] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41: Time spent starting tablet: real 0.024s user 0.007s sys 0.017s
W20260809 12:03:19.834775 1637 tablet.cc:2369] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:03:19.853940 1820 tablet.cc:2369] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:03:19.878180 2001 tablet.cc:2369] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:03:19.987654 2041 raft_consensus.cc:482] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:19.988019 2041 raft_consensus.cc:504] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } }
I20260809 12:03:19.990458 2041 leader_election.cc:288] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f1585337d6246deb49f6eb2c6650851 (127.0.147.66:35161), a15f86377dea492bbe06854dae3958b9 (127.0.147.65:36743)
I20260809 12:03:19.998332 2022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.990903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:42865 (local address 127.0.147.66:35161)
0809 12:03:19.991217 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:19.991225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:19.991267 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:03:19.991352 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:19.991358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:19.991471 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:19.991690 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:19.992360 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.993441 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.996297 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.997346 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.997467 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.997517 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:19.997987 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 12:03:19.998090 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:03:19.998517 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.990805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:35161 (local address 127.0.147.67:42865)
0809 12:03:19.991046 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:19.991070 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:19.991177 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:19.991729 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:19.991736 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:19.991779 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:19.992126 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.992140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.993582 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.993587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.996103 (+ 2516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.996118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.997512 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.997519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.997812 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:19.997850 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:03:19.997965 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 12:03:19.998130 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 12:03:19.999996 1762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02631e23ab014acbb9d92999b52660a1" candidate_uuid: "1ac4457ebbf4473fb533dc57b079af41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1585337d6246deb49f6eb2c6650851" is_pre_election: true
I20260809 12:03:20.001358 1762 raft_consensus.cc:2454] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ac4457ebbf4473fb533dc57b079af41 in term 0.
I20260809 12:03:20.002084 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.992080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:52409 (local address 127.0.147.65:36743)
0809 12:03:19.992380 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:19.992386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:19.993192 (+ 806us) server_negotiation.cc:407] Connection header received
0809 12:03:19.993340 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:19.993345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:19.993423 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:19.993563 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:19.994864 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.995905 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.998997 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.000918 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.001122 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:20.001230 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:20.001565 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 12:03:20.001773 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 12:03:20.002192 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:19.991646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36743 (local address 127.0.147.67:52409)
0809 12:03:19.992989 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:19.993011 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:19.993215 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:19.993582 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:19.993587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:19.993617 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:19.994642 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.994665 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:19.996153 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:19.996161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:19.998782 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:19.998795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.001149 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.001154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:20.001324 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:20.001403 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 12:03:20.001501 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 12:03:20.001605 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"thread_start_us":801,"threads_started":1}
I20260809 12:03:20.002686 1852 leader_election.cc:302] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac4457ebbf4473fb533dc57b079af41, 7f1585337d6246deb49f6eb2c6650851; no voters:
I20260809 12:03:20.003122 2041 raft_consensus.cc:2781] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:20.003316 2041 raft_consensus.cc:482] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:20.003492 2041 raft_consensus.cc:3037] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:20.004134 1580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02631e23ab014acbb9d92999b52660a1" candidate_uuid: "1ac4457ebbf4473fb533dc57b079af41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a15f86377dea492bbe06854dae3958b9" is_pre_election: true
I20260809 12:03:20.004848 1580 raft_consensus.cc:2454] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ac4457ebbf4473fb533dc57b079af41 in term 0.
I20260809 12:03:20.007181 2041 raft_consensus.cc:504] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } }
I20260809 12:03:20.008114 2041 leader_election.cc:288] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [CANDIDATE]: Term 1 election: Requested vote from peers 7f1585337d6246deb49f6eb2c6650851 (127.0.147.66:35161), a15f86377dea492bbe06854dae3958b9 (127.0.147.65:36743)
I20260809 12:03:20.008541 1762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02631e23ab014acbb9d92999b52660a1" candidate_uuid: "1ac4457ebbf4473fb533dc57b079af41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1585337d6246deb49f6eb2c6650851"
I20260809 12:03:20.008608 1580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02631e23ab014acbb9d92999b52660a1" candidate_uuid: "1ac4457ebbf4473fb533dc57b079af41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a15f86377dea492bbe06854dae3958b9"
I20260809 12:03:20.008760 1762 raft_consensus.cc:3037] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:20.008939 1580 raft_consensus.cc:3037] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:20.011940 1762 raft_consensus.cc:2454] T 02631e23ab014acbb9d92999b52660a1 P 7f1585337d6246deb49f6eb2c6650851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ac4457ebbf4473fb533dc57b079af41 in term 1.
I20260809 12:03:20.012244 1580 raft_consensus.cc:2454] T 02631e23ab014acbb9d92999b52660a1 P a15f86377dea492bbe06854dae3958b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ac4457ebbf4473fb533dc57b079af41 in term 1.
I20260809 12:03:20.012465 1852 leader_election.cc:302] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac4457ebbf4473fb533dc57b079af41, 7f1585337d6246deb49f6eb2c6650851; no voters:
I20260809 12:03:20.012846 2041 raft_consensus.cc:2781] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:20.013863 2041 raft_consensus.cc:686] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [term 1 LEADER]: Becoming Leader. State: Replica: 1ac4457ebbf4473fb533dc57b079af41, State: Running, Role: LEADER
I20260809 12:03:20.014671 2041 consensus_queue.cc:237] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } }
I20260809 12:03:20.020794 1376 catalog_manager.cc:5495] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ac4457ebbf4473fb533dc57b079af41 (127.0.147.67). New cstate: current_term: 1 leader_uuid: "1ac4457ebbf4473fb533dc57b079af41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1585337d6246deb49f6eb2c6650851" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ac4457ebbf4473fb533dc57b079af41" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a15f86377dea492bbe06854dae3958b9" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36743 } health_report { overall_health: UNKNOWN } } }
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786276997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:20.050652 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:20.030101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36490 (local address 127.0.147.65:36743)
0809 12:03:20.030335 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:20.030342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:20.031597 (+ 1255us) server_negotiation.cc:407] Connection header received
0809 12:03:20.031780 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:20.031784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:20.031848 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:20.031949 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:20.034557 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.035610 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.037273 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.037930 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.038026 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:20.044004 (+ 5978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:20.044020 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:20.044036 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:20.044071 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:20.047843 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.048515 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.048523 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.048525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.048744 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.049104 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.049108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.049110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.049580 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:20.049876 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:20.050049 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 12:03:20.050215 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277000, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277000, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:20.115272 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:20.095693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.0.147.65:36743)
0809 12:03:20.095913 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:20.095929 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:20.096413 (+ 484us) server_negotiation.cc:407] Connection header received
0809 12:03:20.096640 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:20.096645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:20.096713 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:20.096823 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:20.098224 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.098973 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.100894 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.101363 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.101463 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:20.107908 (+ 6445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:20.107913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:20.107916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:20.107948 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:20.111745 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.112592 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.112596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.112598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.112729 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.113350 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.113353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.113355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.113821 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:20.114279 (+ 458us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:20.114456 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 12:03:20.114736 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:20.174870 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:20.159625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36500 (local address 127.0.147.65:36743)
0809 12:03:20.159895 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:20.159905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:20.160380 (+ 475us) server_negotiation.cc:407] Connection header received
0809 12:03:20.160578 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:20.160582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:20.160651 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:20.160742 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:20.162572 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.163366 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.164718 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.165142 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.165235 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:20.169224 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:20.169230 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:20.169233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:20.169299 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:20.172649 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.173289 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.173292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.173295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.173409 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.173720 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.173723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.173724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.174022 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:20.174203 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:20.174445 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 12:03:20.174622 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260809 12:03:20.176960 1524 tablet_service.cc:2908] Rejecting scan request for tablet 02631e23ab014acbb9d92999b52660a1: Uninitialized: clean time has not yet been initialized
I20260809 12:03:20.180444 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 1460
I20260809 12:03:20.194293 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:20.181447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.0.147.65:36743)
0809 12:03:20.181658 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:20.181664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:20.181889 (+ 225us) server_negotiation.cc:407] Connection header received
0809 12:03:20.181996 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:20.182000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:20.182088 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:20.182161 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:20.183257 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.183986 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.185157 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.185761 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.185918 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:20.188174 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:20.188178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:20.188181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:20.188209 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:20.191744 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.192377 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.192381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.192383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.192568 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.192883 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.192886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.192888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.193292 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:20.193650 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:20.193808 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 12:03:20.193983 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:03:20.194981 1629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:20.285740 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 1460
I20260809 12:03:20.314424 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 1644
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:03:20.329775 2022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:20.315234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60324 (local address 127.0.147.66:35161)
0809 12:03:20.315436 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:20.315442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:20.315464 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:03:20.315715 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:20.315727 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:20.315802 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:20.315911 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:20.317213 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.317941 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.319218 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.319753 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.319913 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:20.324165 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:20.324180 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:20.324194 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:20.324226 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:20.326960 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.327658 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.327668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.327671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.327797 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.328207 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.328213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.328221 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.328591 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:20.328919 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:20.329221 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 12:03:20.329449 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:03:20.330407 1811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:20.408179 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:20.407588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36743 (local address 127.0.147.67:33551)
0809 12:03:20.407775 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:20.407984 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 12:03:20.408500 1852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36743: connect: Connection refused (error 111)
I20260809 12:03:20.410106 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:20.409697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36743 (local address 127.0.147.67:52025)
0809 12:03:20.409863 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:20.409948 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 12:03:20.410385 1852 consensus_peers.cc:487] T 02631e23ab014acbb9d92999b52660a1 P 1ac4457ebbf4473fb533dc57b079af41 -> Peer a15f86377dea492bbe06854dae3958b9 (127.0.147.65:36743): Couldn't send request to peer a15f86377dea492bbe06854dae3958b9. Status: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36743: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 12:03:20.419308 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 1644
I20260809 12:03:20.445722 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 1830
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:03:20.460948 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:20.446651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49282 (local address 127.0.147.67:45031)
0809 12:03:20.447195 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:20.447200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:20.447221 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:03:20.447275 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:20.447278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:20.447341 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:20.447433 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:20.448758 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.449478 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.450694 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:20.451099 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:20.451187 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:20.455494 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:20.455509 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:20.455576 (+ 67us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:20.455620 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:20.458453 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.459078 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.459085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.459087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.459223 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:20.459589 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:20.459593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:20.459595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:20.459914 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:20.460141 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:20.460397 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 12:03:20.460627 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":325,"threads_started":1}
I20260809 12:03:20.461647 1994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:20.553128 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 1830
I20260809 12:03:20.553277 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:20.552725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:35161 (local address 127.0.147.67:50993)
0809 12:03:20.552910 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:20.553068 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.66:35161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 12:03:20.576918 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 1336
I20260809 12:03:20.578140 1416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:20.694530 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 1336
2026-08-09T12:03:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4676 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[2069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[2069](info): set up 2 sockets
Aug 09 12:03:20 dist-test-slave-sb2z krb5kdc[2069](info): commencing operation
krb5kdc: starting...
W20260809 12:03:22.000689 589 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.008s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:03:22 dist-test-slave-sb2z krb5kdc[2069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:03:22.210511 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:33527
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:33527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:22.530779 2097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:22.531067 2097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:22.531325 2097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:22.539155 2097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:22.539284 2097 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 12:03:22.539359 2097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:22.539480 2097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:22.539585 2097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:22.539690 2097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:22.549508 2097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:33527
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:33527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:22.550771 2097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:22.552158 2097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:22.563868 2107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:22.563897 2103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:22.563898 2105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:22.564942 2097 server_base.cc:1034] running on GCE node
I20260809 12:03:22.565657 2097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:22.566859 2097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:22.568099 2097 hybrid_clock.cc:648] HybridClock initialized: now 1786277002568052 us; error 29 us; skew 500 ppm
I20260809 12:03:22.568418 2097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:22 dist-test-slave-sb2z krb5kdc[2069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277002, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:22.571565 2097 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:03:22.573014 2097 webserver.cc:460] Webserver started at http://127.0.147.126:35743/ using document root <none> and password file <none>
I20260809 12:03:22.573733 2097 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:22.573868 2097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:22.574213 2097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:22.577184 2097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "dd8a56a88664496ca4e323fb6ae48b4d"
format_stamp: "Formatted at 2026-08-09 12:03:22 on dist-test-slave-sb2z"
server_key: "8f1bdac1b3206964196d9d3f94ce0912"
server_key_iv: "77bdecb417bac1e6c277f9eaec55c1ab"
server_key_version: "encryptionkey@0"
I20260809 12:03:22.577986 2097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "dd8a56a88664496ca4e323fb6ae48b4d"
format_stamp: "Formatted at 2026-08-09 12:03:22 on dist-test-slave-sb2z"
server_key: "8f1bdac1b3206964196d9d3f94ce0912"
server_key_iv: "77bdecb417bac1e6c277f9eaec55c1ab"
server_key_version: "encryptionkey@0"
I20260809 12:03:22.582974 2097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:03:22.586354 2115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:22.587635 2097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:22.587807 2097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "dd8a56a88664496ca4e323fb6ae48b4d"
format_stamp: "Formatted at 2026-08-09 12:03:22 on dist-test-slave-sb2z"
server_key: "8f1bdac1b3206964196d9d3f94ce0912"
server_key_iv: "77bdecb417bac1e6c277f9eaec55c1ab"
server_key_version: "encryptionkey@0"
I20260809 12:03:22.588030 2097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:22.599644 2097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:22.603328 2097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:22.603708 2097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:22.626144 2097 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:33527
I20260809 12:03:22.626175 2195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:33527 every 8 connection(s)
I20260809 12:03:22.627516 2097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:22.628154 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 2097
I20260809 12:03:22.628399 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:22.629295 589 external_mini_cluster.cc:1428] Setting key a531f0eb990a434e3347b715bee42338
I20260809 12:03:22.631989 2196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:22.642251 2196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d: Bootstrap starting.
Aug 09 12:03:22 dist-test-slave-sb2z krb5kdc[2069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:22.646034 2196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:22.647436 2196 log.cc:824] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:22.650846 2196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d: No bootstrap required, opened a new log
I20260809 12:03:22.652568 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:22.631269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49392 (local address 127.0.147.126:33527)
0809 12:03:22.632127 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:22.632144 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:22.632197 (+ 53us) server_negotiation.cc:407] Connection header received
0809 12:03:22.633800 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:22.633832 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:22.634192 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:22.634595 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:22.636149 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:22.637482 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:22.638732 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:22.639404 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:22.639661 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:22.643902 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:22.643914 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:22.643934 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:22.643992 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:22.648060 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:22.648933 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:22.648942 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:22.648957 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:22.649156 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:22.649506 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:22.649510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:22.649515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:22.650192 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:22.650487 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:22.650809 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 12:03:22.651210 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":414,"threads_started":1}
I20260809 12:03:22.659225 2196 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd8a56a88664496ca4e323fb6ae48b4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33527 } }
I20260809 12:03:22.659643 2196 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:22.659742 2196 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd8a56a88664496ca4e323fb6ae48b4d, State: Initialized, Role: FOLLOWER
I20260809 12:03:22.660331 2196 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd8a56a88664496ca4e323fb6ae48b4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33527 } }
I20260809 12:03:22.660558 2196 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:22.660688 2196 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:22.660861 2196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:22.662919 2196 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd8a56a88664496ca4e323fb6ae48b4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33527 } }
I20260809 12:03:22.663403 2196 leader_election.cc:302] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd8a56a88664496ca4e323fb6ae48b4d; no voters:
I20260809 12:03:22.663954 2196 leader_election.cc:288] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:22.664098 2204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:22.664871 2204 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [term 1 LEADER]: Becoming Leader. State: Replica: dd8a56a88664496ca4e323fb6ae48b4d, State: Running, Role: LEADER
I20260809 12:03:22.665452 2204 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd8a56a88664496ca4e323fb6ae48b4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33527 } }
I20260809 12:03:22.665944 2196 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:22.668666 2205 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd8a56a88664496ca4e323fb6ae48b4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd8a56a88664496ca4e323fb6ae48b4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33527 } } }
I20260809 12:03:22.669104 2205 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:22.669581 2206 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd8a56a88664496ca4e323fb6ae48b4d. Latest consensus state: current_term: 1 leader_uuid: "dd8a56a88664496ca4e323fb6ae48b4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd8a56a88664496ca4e323fb6ae48b4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33527 } } }
I20260809 12:03:22.669854 2206 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:22.672329 2215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:22.677268 2215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:22.686007 2215 catalog_manager.cc:1347] Generated new cluster ID: 2a1b366053ea4f5794b11de6fbf4476f
I20260809 12:03:22.686127 2215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:22.699326 2215 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:22.701186 2215 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:22.716584 2215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d: Generated new TSK 0
I20260809 12:03:22.717614 2215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:03:22.769356 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:33527
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:23.105317 2228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:23.105638 2228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:23.105810 2228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:23.113204 2228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:23.113358 2228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:23.113525 2228 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:03:23.113672 2228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:03:23.123219 2228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.147.126:33527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:23.124493 2228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:23.125828 2228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:23.138983 2234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:23.139055 2237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:23.139065 2235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:23.139971 2228 server_base.cc:1034] running on GCE node
I20260809 12:03:23.140667 2228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:23.141587 2228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:23.142836 2228 hybrid_clock.cc:648] HybridClock initialized: now 1786277003142759 us; error 54 us; skew 500 ppm
I20260809 12:03:23.143175 2228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:23 dist-test-slave-sb2z krb5kdc[2069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:23.146425 2228 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:03:23.147976 2228 webserver.cc:460] Webserver started at http://127.0.147.65:35259/ using document root <none> and password file <none>
I20260809 12:03:23.148677 2228 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:23.148809 2228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:23.149188 2228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:23.152287 2228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "103d12fe7c3b40f281e9bb71b9630b10"
format_stamp: "Formatted at 2026-08-09 12:03:23 on dist-test-slave-sb2z"
server_key: "bb48e243c47e967b6f2bc66840c2973d"
server_key_iv: "58a8e690216d4f1ec6b44de758bb3dce"
server_key_version: "encryptionkey@0"
I20260809 12:03:23.153103 2228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "103d12fe7c3b40f281e9bb71b9630b10"
format_stamp: "Formatted at 2026-08-09 12:03:23 on dist-test-slave-sb2z"
server_key: "bb48e243c47e967b6f2bc66840c2973d"
server_key_iv: "58a8e690216d4f1ec6b44de758bb3dce"
server_key_version: "encryptionkey@0"
I20260809 12:03:23.158215 2228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:03:23.161741 2247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:23.162963 2228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:03:23.163149 2228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "103d12fe7c3b40f281e9bb71b9630b10"
format_stamp: "Formatted at 2026-08-09 12:03:23 on dist-test-slave-sb2z"
server_key: "bb48e243c47e967b6f2bc66840c2973d"
server_key_iv: "58a8e690216d4f1ec6b44de758bb3dce"
server_key_version: "encryptionkey@0"
I20260809 12:03:23.163429 2228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:23.191449 2228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:23.195758 2228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:23.196071 2228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:23.199079 2228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:23.199198 2228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:23.199376 2228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:23.199448 2228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:03:23 dist-test-slave-sb2z krb5kdc[2069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:23.223896 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:23.199480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:45813 (local address 127.0.147.126:33527)
0809 12:03:23.200103 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:23.200112 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:23.201409 (+ 1297us) server_negotiation.cc:407] Connection header received
0809 12:03:23.203572 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:23.203580 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:23.203683 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:23.203822 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:23.206014 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.207319 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.210087 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.210818 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.210975 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:23.216574 (+ 5599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:23.216583 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:23.216588 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:23.216636 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:23.219956 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:23.221239 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:23.221245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:23.221248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:23.221413 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:23.222040 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:23.222046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:23.222049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:23.222347 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:23.222519 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:23.223250 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 12:03:23.223488 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":297,"threads_started":1}
I20260809 12:03:23.225190 2265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:23.200314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:33527 (local address 127.0.147.65:45813)
0809 12:03:23.201245 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:23.201290 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:23.203179 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:23.204009 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:23.204018 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:23.204757 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:23.205713 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:23.205738 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.207503 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.207509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:23.209847 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:23.209860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.211155 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.211167 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:23.211502 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:23.212472 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:23.212508 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:23.216243 (+ 3735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:23.220122 (+ 3879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:23.220133 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:23.220159 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:23.221043 (+ 884us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:23.221582 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:23.221593 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:23.221597 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:23.221885 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:23.222492 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:23.222499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:23.222852 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 12:03:23.223126 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 12:03:23.223674 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":439,"threads_started":1}
I20260809 12:03:23.244854 2228 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:40583
I20260809 12:03:23.244876 2418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:40583 every 8 connection(s)
I20260809 12:03:23.246901 2228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:03:23.250315 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 2228
I20260809 12:03:23.250567 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:03:23.251341 589 external_mini_cluster.cc:1428] Setting key 9162c869ee54bc514501ec426ae8bd17
I20260809 12:03:23.249418 2419 heartbeater.cc:347] Connected to a master server at 127.0.147.126:33527
I20260809 12:03:23.251896 2419 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:23.252373 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:23.233952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:38613 (local address 127.0.147.126:33527)
0809 12:03:23.234173 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:23.234182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:23.236209 (+ 2027us) server_negotiation.cc:407] Connection header received
0809 12:03:23.236427 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:23.236434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:23.236501 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:23.236634 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:23.238383 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.239607 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.242454 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.243093 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.243202 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:23.246069 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:23.246074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:23.246077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:23.246108 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:23.249214 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:23.250132 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:23.250136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:23.250139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:23.250274 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:23.250761 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:23.250765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:23.250767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:23.251041 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:23.251153 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:23.251729 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 12:03:23.251977 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:03:23.252444 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:23.234130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:33527 (local address 127.0.147.65:38613)
0809 12:03:23.236098 (+ 1968us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:23.236114 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:23.236276 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:23.236713 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:23.236719 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:23.237551 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:23.238162 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:23.238182 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.239883 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.239892 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:23.242217 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:23.242242 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.243590 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.243600 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:23.243881 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:23.244703 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:23.244729 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:23.245879 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:23.249387 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:23.249392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:23.249400 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:23.250020 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:23.250390 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:23.250397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:23.250404 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:23.250603 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:23.251170 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:23.251177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:23.251386 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 12:03:23.251604 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 12:03:23.251936 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1801,"thread_start_us":1567,"threads_started":1}
I20260809 12:03:23.252851 2419 heartbeater.cc:511] Master 127.0.147.126:33527 requested a full tablet report, sending...
I20260809 12:03:23.254796 2133 ts_manager.cc:194] Registered new tserver with Master: 103d12fe7c3b40f281e9bb71b9630b10 (127.0.147.65:40583)
I20260809 12:03:23.256304 2133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:45813
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:03:23.298497 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:33527
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:23.631323 2425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:23.631698 2425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:23.631876 2425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:23.639111 2425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:23.639259 2425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:23.639420 2425 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:03:23.639616 2425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:03:23.649713 2425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.147.126:33527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:23.650986 2425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:23.652402 2425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:23.665441 2431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:23.665419 2432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:23.665474 2435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:23.666635 2425 server_base.cc:1034] running on GCE node
I20260809 12:03:23.667438 2425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:23.668476 2425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:23.669724 2425 hybrid_clock.cc:648] HybridClock initialized: now 1786277003669657 us; error 45 us; skew 500 ppm
I20260809 12:03:23.670082 2425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:23 dist-test-slave-sb2z krb5kdc[2069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:23.673396 2425 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:03:23.674729 2425 webserver.cc:460] Webserver started at http://127.0.147.66:38027/ using document root <none> and password file <none>
I20260809 12:03:23.675382 2425 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:23.675472 2425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:23.675802 2425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:23.678589 2425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "6ff2488bac874be5962e011227e7b515"
format_stamp: "Formatted at 2026-08-09 12:03:23 on dist-test-slave-sb2z"
server_key: "bbd8c39239f6c8d7750225a49ba9921a"
server_key_iv: "b4c8323890124e0847fd39545309d709"
server_key_version: "encryptionkey@0"
I20260809 12:03:23.679365 2425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "6ff2488bac874be5962e011227e7b515"
format_stamp: "Formatted at 2026-08-09 12:03:23 on dist-test-slave-sb2z"
server_key: "bbd8c39239f6c8d7750225a49ba9921a"
server_key_iv: "b4c8323890124e0847fd39545309d709"
server_key_version: "encryptionkey@0"
I20260809 12:03:23.684515 2425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:03:23.687990 2445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:23.689342 2425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:03:23.689479 2425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "6ff2488bac874be5962e011227e7b515"
format_stamp: "Formatted at 2026-08-09 12:03:23 on dist-test-slave-sb2z"
server_key: "bbd8c39239f6c8d7750225a49ba9921a"
server_key_iv: "b4c8323890124e0847fd39545309d709"
server_key_version: "encryptionkey@0"
I20260809 12:03:23.689668 2425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:23.709643 2425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:23.714018 2425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:23.714251 2425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:23.717031 2425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:23.717154 2425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:23.717365 2425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:23.717450 2425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:03:23 dist-test-slave-sb2z krb5kdc[2069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:23.742184 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:23.717434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:34689 (local address 127.0.147.126:33527)
0809 12:03:23.717610 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:23.717614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:23.719127 (+ 1513us) server_negotiation.cc:407] Connection header received
0809 12:03:23.721287 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:23.721294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:23.721395 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:23.721520 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:23.723547 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.724665 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.727154 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.728054 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.728213 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:23.734621 (+ 6408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:23.734630 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:23.734637 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:23.734689 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:23.738444 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:23.739206 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:23.739213 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:23.739217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:23.739355 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:23.739918 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:23.739926 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:23.739930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:23.740247 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:23.740425 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:23.741561 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0809 12:03:23.741805 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 12:03:23.743804 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:23.718115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:33527 (local address 127.0.147.66:34689)
0809 12:03:23.718959 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:23.719002 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:23.720928 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:23.721722 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:23.721729 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:23.722404 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:23.723345 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:23.723368 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.724843 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.724850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:23.726972 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:23.726992 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.728715 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.728727 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:23.729074 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:23.729856 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:23.729902 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:23.734261 (+ 4359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:23.738605 (+ 4344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:23.738614 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:23.738634 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:23.739075 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:23.739477 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:23.739482 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:23.739485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:23.739789 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:23.740550 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:23.740560 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:23.741068 (+ 508us) client_negotiation.cc:769] Sending connection context
0809 12:03:23.741470 (+ 402us) client_negotiation.cc:240] Negotiation successful
0809 12:03:23.742108 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":365,"threads_started":1}
I20260809 12:03:23.762802 2621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:32965 every 8 connection(s)
I20260809 12:03:23.762836 2425 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:32965
I20260809 12:03:23.764602 2425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:03:23.767339 2622 heartbeater.cc:347] Connected to a master server at 127.0.147.126:33527
I20260809 12:03:23.767719 2584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:23.751830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:33527 (local address 127.0.147.66:51185)
0809 12:03:23.752513 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:23.752533 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:23.752734 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:23.753259 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:23.753265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:23.753949 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:23.754476 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:23.754496 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.755970 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.755975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:23.757532 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:23.757546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.758492 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.758500 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:23.758670 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:23.759499 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:23.759574 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:23.760414 (+ 840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:23.764237 (+ 3823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:23.764240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:23.764247 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:23.764702 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:23.765172 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:23.765177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:23.765180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:23.765388 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:23.766309 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:23.766316 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:23.766528 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 12:03:23.766703 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 12:03:23.767068 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":382,"threads_started":1}
I20260809 12:03:23.767727 2622 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:23.767989 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:23.752136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:51185 (local address 127.0.147.126:33527)
0809 12:03:23.752333 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:23.752339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:23.752661 (+ 322us) server_negotiation.cc:407] Connection header received
0809 12:03:23.752889 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:23.752896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:23.752985 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:23.753094 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:23.754658 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.755817 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.757694 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:23.758286 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:23.758425 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:23.760663 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:23.760671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:23.760681 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:23.760726 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:23.764105 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:23.764848 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:23.764855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:23.764858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:23.765021 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:23.765575 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:23.765580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:23.765593 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:23.766082 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:23.766238 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:23.767340 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0809 12:03:23.767635 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:03:23.767969 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 2425
I20260809 12:03:23.768224 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:03:23.768723 589 external_mini_cluster.cc:1428] Setting key 91f2e9b813dce2fd5f280f8eb183b830
I20260809 12:03:23.768985 2622 heartbeater.cc:511] Master 127.0.147.126:33527 requested a full tablet report, sending...
I20260809 12:03:23.770579 2134 ts_manager.cc:194] Registered new tserver with Master: 6ff2488bac874be5962e011227e7b515 (127.0.147.66:32965)
I20260809 12:03:23.771454 2134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:34689
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:03:23.819581 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:33527
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:24.162022 2628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:24.162350 2628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:24.162537 2628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:24.170240 2628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:24.170384 2628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:24.170552 2628 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:03:24.170691 2628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:03:24.180332 2628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.147.126:33527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:24.181649 2628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:24.183027 2628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:24.196367 2636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:24.196323 2637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:24.196400 2640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:24.197988 2628 server_base.cc:1034] running on GCE node
I20260809 12:03:24.198693 2628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:24.199746 2628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:24.200974 2628 hybrid_clock.cc:648] HybridClock initialized: now 1786277004200917 us; error 46 us; skew 500 ppm
I20260809 12:03:24.201313 2628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:24 dist-test-slave-sb2z krb5kdc[2069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277004, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:24.204744 2628 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:03:24.206174 2628 webserver.cc:460] Webserver started at http://127.0.147.67:43277/ using document root <none> and password file <none>
I20260809 12:03:24.206889 2628 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:24.207017 2628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:24.207363 2628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:24.210343 2628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "1f3fefa9119c42459b0c0bf7aa1d7584"
format_stamp: "Formatted at 2026-08-09 12:03:24 on dist-test-slave-sb2z"
server_key: "2a47e1062d9286bcdb026e98228c9475"
server_key_iv: "74734e2f55c0378533e1ac294232ea10"
server_key_version: "encryptionkey@0"
I20260809 12:03:24.211169 2628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "1f3fefa9119c42459b0c0bf7aa1d7584"
format_stamp: "Formatted at 2026-08-09 12:03:24 on dist-test-slave-sb2z"
server_key: "2a47e1062d9286bcdb026e98228c9475"
server_key_iv: "74734e2f55c0378533e1ac294232ea10"
server_key_version: "encryptionkey@0"
I20260809 12:03:24.216354 2628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 12:03:24.219751 2647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:24.221057 2628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:03:24.221228 2628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "1f3fefa9119c42459b0c0bf7aa1d7584"
format_stamp: "Formatted at 2026-08-09 12:03:24 on dist-test-slave-sb2z"
server_key: "2a47e1062d9286bcdb026e98228c9475"
server_key_iv: "74734e2f55c0378533e1ac294232ea10"
server_key_version: "encryptionkey@0"
I20260809 12:03:24.221451 2628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:24.253542 2628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:24.257984 2628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:24.258275 2628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:24.258837 2419 heartbeater.cc:503] Master 127.0.147.126:33527 was elected leader, sending a full tablet report...
I20260809 12:03:24.262022 2628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:24.262141 2628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:24.262324 2628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:24.262405 2628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:03:24 dist-test-slave-sb2z krb5kdc[2069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277004, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:24.286048 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.261987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:36567 (local address 127.0.147.126:33527)
0809 12:03:24.262214 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:24.262220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:24.264162 (+ 1942us) server_negotiation.cc:407] Connection header received
0809 12:03:24.266266 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:24.266272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:24.266362 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:24.266501 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:24.268811 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.269873 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.272681 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.273310 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.273458 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.278627 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:24.278636 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:24.278642 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:24.278692 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:24.282582 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:24.283610 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:24.283616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:24.283620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:24.283766 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:24.284237 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:24.284243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:24.284246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:24.284533 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:24.284700 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:24.285443 (+ 743us) server_negotiation.cc:299] Negotiation successful
0809 12:03:24.285691 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:03:24.287130 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.262775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:33527 (local address 127.0.147.67:36567)
0809 12:03:24.263984 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:24.264029 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:24.265948 (+ 1919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:24.266797 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:24.266811 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:24.267621 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:24.268535 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.268573 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.270084 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.270087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.272519 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.272531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.273660 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.273675 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.274044 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.274922 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:24.274945 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:24.278333 (+ 3388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:24.282722 (+ 4389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:24.282731 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:24.282748 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:24.283443 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:24.283884 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:24.283890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:24.283894 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:24.284114 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:24.284685 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:24.284693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:24.285037 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 12:03:24.285307 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 12:03:24.285837 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":582,"threads_started":1}
I20260809 12:03:24.314670 2628 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:43651
I20260809 12:03:24.314714 2818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:43651 every 8 connection(s)
I20260809 12:03:24.315400 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.297471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:33527 (local address 127.0.147.67:58531)
0809 12:03:24.298186 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:24.298211 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:24.298400 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:24.298850 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:24.298854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:24.299300 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:24.299710 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.299728 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.301543 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.301551 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.303775 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.303796 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.305021 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.305030 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.305241 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.305929 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:24.305946 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:24.307461 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:24.312209 (+ 4748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:24.312218 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:24.312222 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:24.312798 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:24.313269 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:24.313276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:24.313281 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:24.313481 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:24.314248 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:24.314252 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:24.314502 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 12:03:24.314683 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 12:03:24.315074 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":383,"threads_started":1}
I20260809 12:03:24.315418 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.297461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:58531 (local address 127.0.147.126:33527)
0809 12:03:24.297725 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:24.297738 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:24.298323 (+ 585us) server_negotiation.cc:407] Connection header received
0809 12:03:24.298558 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:24.298567 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:24.298676 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:24.298806 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:24.299880 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.301282 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.303983 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.304771 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.304974 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.307758 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:24.307767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:24.307776 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:24.307834 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:24.312042 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:24.312947 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:24.312953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:24.312957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:24.313126 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:24.313639 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:24.313645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:24.313650 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:24.314109 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:24.314284 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:24.314839 (+ 555us) server_negotiation.cc:299] Negotiation successful
0809 12:03:24.315100 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 12:03:24.316500 2628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:03:24.318980 2820 heartbeater.cc:347] Connected to a master server at 127.0.147.126:33527
I20260809 12:03:24.319648 2820 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:24.320318 2820 heartbeater.cc:511] Master 127.0.147.126:33527 requested a full tablet report, sending...
I20260809 12:03:24.321730 2134 ts_manager.cc:194] Registered new tserver with Master: 1f3fefa9119c42459b0c0bf7aa1d7584 (127.0.147.67:43651)
I20260809 12:03:24.322538 2134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:36567
I20260809 12:03:24.323302 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 2628
I20260809 12:03:24.323559 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:03:24.324024 589 external_mini_cluster.cc:1428] Setting key 006dcb2c07b8ac96f12844b208a6be5f
I20260809 12:03:24.327049 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:03:24 dist-test-slave-sb2z krb5kdc[2069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:03:24 dist-test-slave-sb2z krb5kdc[2069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:24.392606 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.378139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49402 (local address 127.0.147.126:33527)
0809 12:03:24.378387 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:24.378394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:24.378646 (+ 252us) server_negotiation.cc:407] Connection header received
0809 12:03:24.378861 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:24.378867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:24.378941 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:24.379050 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:24.380541 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.381313 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.382622 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.383061 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.383206 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.387116 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:24.387121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:24.387125 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:24.387157 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:24.389713 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:24.390376 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:24.390382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:24.390387 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:24.390539 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:24.390926 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:24.390931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:24.390936 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:24.391278 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:24.391587 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:24.391910 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 12:03:24.392146 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:03:24.396791 2134 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:03:24.403705 2134 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:24.433419 2835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.420831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:32965 (local address 127.0.0.1:57250)
0809 12:03:24.423151 (+ 2320us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:24.423175 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:24.423325 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:24.424840 (+ 1515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:24.424857 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:24.424897 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:24.425391 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.425405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.427734 (+ 2329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.427740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.430588 (+ 2848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.430603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.432358 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.432364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.432568 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.432623 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 12:03:24.432761 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 12:03:24.432975 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2145,"mutex_wait_us":499,"thread_start_us":1953,"threads_started":1}
I20260809 12:03:24.433322 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.421392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.0.147.66:32965)
0809 12:03:24.423975 (+ 2583us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:24.423988 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:24.424040 (+ 52us) server_negotiation.cc:407] Connection header received
0809 12:03:24.424167 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:24.424179 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:24.424523 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:24.425738 (+ 1215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:24.426059 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.427596 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.430827 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.432130 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.432314 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.432540 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:24.432819 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 12:03:24.432998 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2361,"thread_start_us":2213,"threads_started":1}
I20260809 12:03:24.434454 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.420946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35410 (local address 127.0.147.67:43651)
0809 12:03:24.425559 (+ 4613us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:24.425567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:24.425602 (+ 35us) server_negotiation.cc:407] Connection header received
0809 12:03:24.425707 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:24.425715 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:24.425932 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:24.427000 (+ 1068us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:24.427422 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.429031 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.431765 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.433266 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.433660 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.433896 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:24.433962 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 12:03:24.434122 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4112,"thread_start_us":3907,"threads_started":1}
I20260809 12:03:24.434752 2836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.420751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43651 (local address 127.0.0.1:35410)
0809 12:03:24.421663 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:24.421715 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:24.421947 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:24.426165 (+ 4218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:24.426170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:24.426198 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:24.427113 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.427127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.429205 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.429212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.431589 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.431604 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.433500 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.433506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.433706 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.433740 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:03:24.433826 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 12:03:24.434282 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":19,"thread_start_us":387,"threads_started":1}
I20260809 12:03:24.435285 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.422496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40583 (local address 127.0.0.1:57596)
0809 12:03:24.424864 (+ 2368us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:24.424886 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:24.425046 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:24.426066 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:24.426071 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:24.426094 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:24.426759 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.426769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.428722 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.428728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.431179 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.431190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.433436 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.433466 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.434647 (+ 1181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.434683 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:03:24.434796 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 12:03:24.434935 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2162,"mutex_wait_us":103,"thread_start_us":2012,"threads_started":1}
I20260809 12:03:24.436122 2843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.423086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57596 (local address 127.0.147.65:40583)
0809 12:03:24.424305 (+ 1219us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:24.424320 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:24.425146 (+ 826us) server_negotiation.cc:407] Connection header received
0809 12:03:24.425262 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:24.425271 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:24.425545 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:24.425773 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:24.426949 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.428578 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.431344 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.432903 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.433050 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.433249 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:24.435625 (+ 2376us) server_negotiation.cc:299] Negotiation successful
0809 12:03:24.435818 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":865,"threads_started":1}
I20260809 12:03:24.438510 2718 tablet_service.cc:1467] Processing CreateTablet for tablet fee1100c46aa4af484e325a7f0dc14a6 (DEFAULT_TABLE table=test-table [id=f96acf52afcb49cbaa78fab441ec47d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:24.438730 2521 tablet_service.cc:1467] Processing CreateTablet for tablet fee1100c46aa4af484e325a7f0dc14a6 (DEFAULT_TABLE table=test-table [id=f96acf52afcb49cbaa78fab441ec47d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:24.439494 2320 tablet_service.cc:1467] Processing CreateTablet for tablet fee1100c46aa4af484e325a7f0dc14a6 (DEFAULT_TABLE table=test-table [id=f96acf52afcb49cbaa78fab441ec47d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:24.440137 2718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee1100c46aa4af484e325a7f0dc14a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:24.440924 2320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee1100c46aa4af484e325a7f0dc14a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:24.441735 2521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee1100c46aa4af484e325a7f0dc14a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:24.458782 2844 tablet_bootstrap.cc:492] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: Bootstrap starting.
I20260809 12:03:24.464556 2844 tablet_bootstrap.cc:654] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:24.466921 2844 log.cc:824] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:24.467767 2846 tablet_bootstrap.cc:492] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: Bootstrap starting.
I20260809 12:03:24.472852 2847 tablet_bootstrap.cc:492] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: Bootstrap starting.
I20260809 12:03:24.475946 2846 tablet_bootstrap.cc:654] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:24.478268 2847 tablet_bootstrap.cc:654] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:24.478389 2844 tablet_bootstrap.cc:492] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: No bootstrap required, opened a new log
I20260809 12:03:24.478349 2846 log.cc:824] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:24.478760 2844 ts_tablet_manager.cc:1397] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.004s
I20260809 12:03:24.480947 2847 log.cc:824] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:24.484246 2846 tablet_bootstrap.cc:492] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: No bootstrap required, opened a new log
I20260809 12:03:24.484709 2846 ts_tablet_manager.cc:1397] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.008s
I20260809 12:03:24.485256 2847 tablet_bootstrap.cc:492] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: No bootstrap required, opened a new log
I20260809 12:03:24.485613 2847 ts_tablet_manager.cc:1397] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260809 12:03:24.494448 2844 raft_consensus.cc:348] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } }
I20260809 12:03:24.495158 2844 raft_consensus.cc:374] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:24.495384 2844 raft_consensus.cc:729] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3fefa9119c42459b0c0bf7aa1d7584, State: Initialized, Role: FOLLOWER
I20260809 12:03:24.496357 2844 consensus_queue.cc:260] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } }
I20260809 12:03:24.501111 2847 raft_consensus.cc:348] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } }
I20260809 12:03:24.501467 2846 raft_consensus.cc:348] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } }
I20260809 12:03:24.501991 2847 raft_consensus.cc:374] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:24.502187 2847 raft_consensus.cc:729] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff2488bac874be5962e011227e7b515, State: Initialized, Role: FOLLOWER
I20260809 12:03:24.502317 2846 raft_consensus.cc:374] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:24.502514 2846 raft_consensus.cc:729] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d12fe7c3b40f281e9bb71b9630b10, State: Initialized, Role: FOLLOWER
I20260809 12:03:24.503156 2847 consensus_queue.cc:260] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } }
I20260809 12:03:24.503773 2846 consensus_queue.cc:260] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } }
I20260809 12:03:24.505312 2820 heartbeater.cc:503] Master 127.0.147.126:33527 was elected leader, sending a full tablet report...
I20260809 12:03:24.506922 2844 ts_tablet_manager.cc:1428] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: Time spent starting tablet: real 0.028s user 0.012s sys 0.013s
I20260809 12:03:24.509208 2846 ts_tablet_manager.cc:1428] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: Time spent starting tablet: real 0.024s user 0.018s sys 0.006s
I20260809 12:03:24.510834 2622 heartbeater.cc:503] Master 127.0.147.126:33527 was elected leader, sending a full tablet report...
I20260809 12:03:24.511803 2847 ts_tablet_manager.cc:1428] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: Time spent starting tablet: real 0.026s user 0.022s sys 0.004s
W20260809 12:03:24.516714 2625 tablet.cc:2369] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:03:24.569286 2821 tablet.cc:2369] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:03:24.705215 2854 raft_consensus.cc:482] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:24.705755 2854 raft_consensus.cc:504] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } }
I20260809 12:03:24.708047 2854 leader_election.cc:288] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f3fefa9119c42459b0c0bf7aa1d7584 (127.0.147.67:43651), 103d12fe7c3b40f281e9bb71b9630b10 (127.0.147.65:40583)
I20260809 12:03:24.716727 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.708455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:34549 (local address 127.0.147.67:43651)
0809 12:03:24.708749 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:24.708760 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:24.709638 (+ 878us) server_negotiation.cc:407] Connection header received
0809 12:03:24.709915 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:24.709921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:24.710037 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:24.710180 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:24.711095 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.712096 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.714411 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.715406 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.715643 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.715763 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:24.716286 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 12:03:24.716413 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:03:24.717351 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.708299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43651 (local address 127.0.147.66:34549)
0809 12:03:24.709447 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:24.709465 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:24.709682 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:24.710247 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:24.710256 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:24.710302 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:24.710913 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.710926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.712239 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.712245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.714200 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.714212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.715873 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.715889 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.716092 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.716117 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:03:24.716260 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 12:03:24.716392 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"thread_start_us":759,"threads_started":1}
I20260809 12:03:24.717839 2843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.709660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:34453 (local address 127.0.147.65:40583)
0809 12:03:24.710036 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:24.710045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:24.710759 (+ 714us) server_negotiation.cc:407] Connection header received
0809 12:03:24.710988 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:24.710992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:24.711052 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:24.711151 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:24.712041 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.713045 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.715435 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.716603 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.716793 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.716900 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:24.717335 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 12:03:24.717496 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:03:24.718079 2858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.709488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40583 (local address 127.0.147.66:34453)
0809 12:03:24.710644 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:24.710659 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:24.710825 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:24.711165 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:24.711169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:24.711194 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:24.711867 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.711879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.713215 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.713219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.715227 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:24.715239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.716900 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.716907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:24.717153 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.717191 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:03:24.717302 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 12:03:24.717434 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"thread_start_us":795,"threads_started":1}
I20260809 12:03:24.718632 2754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee1100c46aa4af484e325a7f0dc14a6" candidate_uuid: "6ff2488bac874be5962e011227e7b515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" is_pre_election: true
I20260809 12:03:24.719209 2346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee1100c46aa4af484e325a7f0dc14a6" candidate_uuid: "6ff2488bac874be5962e011227e7b515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d12fe7c3b40f281e9bb71b9630b10" is_pre_election: true
I20260809 12:03:24.719651 2754 raft_consensus.cc:2454] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff2488bac874be5962e011227e7b515 in term 0.
I20260809 12:03:24.720036 2346 raft_consensus.cc:2454] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff2488bac874be5962e011227e7b515 in term 0.
I20260809 12:03:24.720534 2448 leader_election.cc:302] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f3fefa9119c42459b0c0bf7aa1d7584, 6ff2488bac874be5962e011227e7b515; no voters:
I20260809 12:03:24.721024 2854 raft_consensus.cc:2781] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:24.721221 2854 raft_consensus.cc:482] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:24.721362 2854 raft_consensus.cc:3037] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Advancing to term 1
W20260809 12:03:24.751690 2420 tablet.cc:2369] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:03:24.801532 2854 raft_consensus.cc:504] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } }
I20260809 12:03:24.802732 2854 leader_election.cc:288] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [CANDIDATE]: Term 1 election: Requested vote from peers 1f3fefa9119c42459b0c0bf7aa1d7584 (127.0.147.67:43651), 103d12fe7c3b40f281e9bb71b9630b10 (127.0.147.65:40583)
I20260809 12:03:24.803287 2754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee1100c46aa4af484e325a7f0dc14a6" candidate_uuid: "6ff2488bac874be5962e011227e7b515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584"
I20260809 12:03:24.803596 2754 raft_consensus.cc:3037] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:24.803582 2346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee1100c46aa4af484e325a7f0dc14a6" candidate_uuid: "6ff2488bac874be5962e011227e7b515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d12fe7c3b40f281e9bb71b9630b10"
I20260809 12:03:24.803864 2346 raft_consensus.cc:3037] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:24.806468 2754 raft_consensus.cc:2454] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff2488bac874be5962e011227e7b515 in term 1.
I20260809 12:03:24.807068 2448 leader_election.cc:302] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f3fefa9119c42459b0c0bf7aa1d7584, 6ff2488bac874be5962e011227e7b515; no voters:
I20260809 12:03:24.807442 2854 raft_consensus.cc:2781] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:24.808406 2854 raft_consensus.cc:686] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 1 LEADER]: Becoming Leader. State: Replica: 6ff2488bac874be5962e011227e7b515, State: Running, Role: LEADER
I20260809 12:03:24.808652 2346 raft_consensus.cc:2454] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff2488bac874be5962e011227e7b515 in term 1.
I20260809 12:03:24.809490 2854 consensus_queue.cc:237] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } }
I20260809 12:03:24.815891 2132 catalog_manager.cc:5495] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ff2488bac874be5962e011227e7b515 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "6ff2488bac874be5962e011227e7b515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } health_report { overall_health: UNKNOWN } } }
I20260809 12:03:24.910614 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:24.902973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57254 (local address 127.0.147.66:32965)
0809 12:03:24.903169 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:24.903174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:24.904168 (+ 994us) server_negotiation.cc:407] Connection header received
0809 12:03:24.904556 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:24.904561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:24.904654 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:24.904807 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:24.905476 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.906207 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.907956 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:24.908561 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:24.908715 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:24.909230 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:24.909838 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:24.910018 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:24.910171 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 12:03:24.910258 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:03:24.921046 2346 raft_consensus.cc:1261] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 FOLLOWER]: Refusing update from remote peer 6ff2488bac874be5962e011227e7b515: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:24.922246 2859 consensus_queue.cc:1035] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:24.923640 2754 raft_consensus.cc:1261] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 1 FOLLOWER]: Refusing update from remote peer 6ff2488bac874be5962e011227e7b515: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:24.924484 2859 consensus_queue.cc:1035] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:24.971385 2133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:24.971839 2133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:24.976284 2133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d: Sending DeleteTablet for 3 replicas of tablet fee1100c46aa4af484e325a7f0dc14a6
I20260809 12:03:24.978044 2718 tablet_service.cc:1514] Processing DeleteTablet for tablet fee1100c46aa4af484e325a7f0dc14a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:24 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:35410
I20260809 12:03:24.978096 2521 tablet_service.cc:1514] Processing DeleteTablet for tablet fee1100c46aa4af484e325a7f0dc14a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:24 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:57250
I20260809 12:03:24.978304 2320 tablet_service.cc:1514] Processing DeleteTablet for tablet fee1100c46aa4af484e325a7f0dc14a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:24 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:57596
I20260809 12:03:24.979888 2881 tablet_replica.cc:330] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: stopping tablet replica
I20260809 12:03:24.979962 2880 tablet_replica.cc:330] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: stopping tablet replica
I20260809 12:03:24.980566 2880 raft_consensus.cc:2229] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:24.980568 2881 raft_consensus.cc:2229] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:03:24.981544 2880 raft_consensus.cc:2258] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:24.981678 2133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 12:03:24.982051 2881 raft_consensus.cc:2258] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 12:03:24.982468 2133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:24.985194 2880 ts_tablet_manager.cc:1905] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:24.990607 2881 ts_tablet_manager.cc:1905] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:24.991281 2882 tablet_replica.cc:330] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: stopping tablet replica
I20260809 12:03:24.991840 2882 raft_consensus.cc:2229] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:24.992609 2882 raft_consensus.cc:2258] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:24.993459 2319 tablet_service.cc:1467] Processing CreateTablet for tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc (DEFAULT_TABLE table=test-table [id=43885e216a404d67831ed4184a6f97cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:24.993706 2718 tablet_service.cc:1467] Processing CreateTablet for tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc (DEFAULT_TABLE table=test-table [id=43885e216a404d67831ed4184a6f97cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:24.994094 2319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:24.994387 2718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:24.994964 2882 ts_tablet_manager.cc:1905] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:24.999989 2521 tablet_service.cc:1467] Processing CreateTablet for tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc (DEFAULT_TABLE table=test-table [id=43885e216a404d67831ed4184a6f97cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:25.004411 2846 tablet_bootstrap.cc:492] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: Bootstrap starting.
I20260809 12:03:25.007507 2846 tablet_bootstrap.cc:654] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:25.007507 2521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:25.010885 2844 tablet_bootstrap.cc:492] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: Bootstrap starting.
I20260809 12:03:25.013863 2844 tablet_bootstrap.cc:654] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:25.018723 2846 tablet_bootstrap.cc:492] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: No bootstrap required, opened a new log
I20260809 12:03:25.019038 2846 ts_tablet_manager.cc:1397] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260809 12:03:25.020803 2846 raft_consensus.cc:348] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.021421 2846 raft_consensus.cc:374] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:25.021828 2846 raft_consensus.cc:729] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d12fe7c3b40f281e9bb71b9630b10, State: Initialized, Role: FOLLOWER
I20260809 12:03:25.022432 2846 consensus_queue.cc:260] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.023353 2846 ts_tablet_manager.cc:1428] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 12:03:25.025717 2844 tablet_bootstrap.cc:492] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: No bootstrap required, opened a new log
I20260809 12:03:25.026074 2844 ts_tablet_manager.cc:1397] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260809 12:03:25.025678 2886 tablet_bootstrap.cc:492] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: Bootstrap starting.
I20260809 12:03:25.028438 2844 raft_consensus.cc:348] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.028760 2844 raft_consensus.cc:374] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:25.029089 2844 raft_consensus.cc:729] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3fefa9119c42459b0c0bf7aa1d7584, State: Initialized, Role: FOLLOWER
I20260809 12:03:25.029592 2844 consensus_queue.cc:260] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.030376 2844 ts_tablet_manager.cc:1428] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 12:03:25.030638 2886 tablet_bootstrap.cc:654] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:25.035094 2886 tablet_bootstrap.cc:492] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: No bootstrap required, opened a new log
I20260809 12:03:25.035346 2886 ts_tablet_manager.cc:1397] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 12:03:25.036554 2886 raft_consensus.cc:348] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.036863 2886 raft_consensus.cc:374] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:25.037024 2886 raft_consensus.cc:729] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff2488bac874be5962e011227e7b515, State: Initialized, Role: FOLLOWER
I20260809 12:03:25.037392 2886 consensus_queue.cc:260] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.038122 2886 ts_tablet_manager.cc:1428] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 12:03:25.104218 2880 ts_tablet_manager.cc:1918] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:03:25.104202 2881 ts_tablet_manager.cc:1918] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:03:25.104530 2880 log.cc:1196] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/fee1100c46aa4af484e325a7f0dc14a6
I20260809 12:03:25.104609 2881 log.cc:1196] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/fee1100c46aa4af484e325a7f0dc14a6
I20260809 12:03:25.105196 2880 ts_tablet_manager.cc:1939] T fee1100c46aa4af484e325a7f0dc14a6 P 1f3fefa9119c42459b0c0bf7aa1d7584: Deleting consensus metadata
I20260809 12:03:25.106285 2881 ts_tablet_manager.cc:1939] T fee1100c46aa4af484e325a7f0dc14a6 P 6ff2488bac874be5962e011227e7b515: Deleting consensus metadata
I20260809 12:03:25.108044 2882 ts_tablet_manager.cc:1918] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:03:25.108336 2882 log.cc:1196] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/fee1100c46aa4af484e325a7f0dc14a6
I20260809 12:03:25.109073 2882 ts_tablet_manager.cc:1939] T fee1100c46aa4af484e325a7f0dc14a6 P 103d12fe7c3b40f281e9bb71b9630b10: Deleting consensus metadata
I20260809 12:03:25.109236 2117 catalog_manager.cc:4841] TS 1f3fefa9119c42459b0c0bf7aa1d7584 (127.0.147.67:43651): tablet fee1100c46aa4af484e325a7f0dc14a6 (table test-table [id=f96acf52afcb49cbaa78fab441ec47d6]) successfully deleted
I20260809 12:03:25.109594 2119 catalog_manager.cc:4841] TS 6ff2488bac874be5962e011227e7b515 (127.0.147.66:32965): tablet fee1100c46aa4af484e325a7f0dc14a6 (table test-table [id=f96acf52afcb49cbaa78fab441ec47d6]) successfully deleted
I20260809 12:03:25.158524 2117 catalog_manager.cc:4841] TS 103d12fe7c3b40f281e9bb71b9630b10 (127.0.147.65:40583): tablet fee1100c46aa4af484e325a7f0dc14a6 (table test-table [id=f96acf52afcb49cbaa78fab441ec47d6]) successfully deleted
I20260809 12:03:25.343288 2856 raft_consensus.cc:482] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:25.343652 2856 raft_consensus.cc:504] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.346134 2856 leader_election.cc:288] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ff2488bac874be5962e011227e7b515 (127.0.147.66:32965), 1f3fefa9119c42459b0c0bf7aa1d7584 (127.0.147.67:43651)
I20260809 12:03:25.359329 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:25.346925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:54343 (local address 127.0.147.67:43651)
0809 12:03:25.349243 (+ 2318us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:25.349248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:25.349263 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:03:25.349317 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:25.349321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:25.349376 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:25.349804 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:25.351585 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.353095 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.356217 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.357710 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.358010 (+ 300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:25.358109 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:25.358457 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 12:03:25.358674 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2111,"thread_start_us":1905,"threads_started":1}
I20260809 12:03:25.359385 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:25.346431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43651 (local address 127.0.147.65:54343)
0809 12:03:25.348078 (+ 1647us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:25.348114 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:25.348311 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:25.349543 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:25.349546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:25.349564 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:25.351287 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:25.351302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.353256 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.353264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:25.356052 (+ 2788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:25.356066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.357932 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.357939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:25.358263 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:25.358298 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:03:25.358409 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 12:03:25.358608 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1384,"mutex_wait_us":18,"thread_start_us":1759,"threads_started":1}
I20260809 12:03:25.360423 2754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1f2b42b5da4afa8f1b2a9a2d32e7cc" candidate_uuid: "103d12fe7c3b40f281e9bb71b9630b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" is_pre_election: true
I20260809 12:03:25.360746 2754 raft_consensus.cc:2454] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d12fe7c3b40f281e9bb71b9630b10 in term 0.
I20260809 12:03:25.361260 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:25.346711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:52169 (local address 127.0.147.66:32965)
0809 12:03:25.346969 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:25.346979 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:25.348859 (+ 1880us) server_negotiation.cc:407] Connection header received
0809 12:03:25.349205 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:25.349212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:25.349351 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:25.349451 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:25.350933 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.352616 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.357426 (+ 4810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.359588 (+ 2162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.359798 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:25.359883 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:25.360472 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 12:03:25.360700 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 12:03:25.361392 2253 leader_election.cc:302] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d12fe7c3b40f281e9bb71b9630b10, 1f3fefa9119c42459b0c0bf7aa1d7584; no voters:
I20260809 12:03:25.362000 2856 raft_consensus.cc:2781] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:25.362290 2856 raft_consensus.cc:482] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:25.362991 2856 raft_consensus.cc:3037] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:25.363124 2889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:25.346565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:32965 (local address 127.0.147.65:52169)
0809 12:03:25.348726 (+ 2161us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:25.348755 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:25.348983 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:25.349502 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:25.349510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:25.349551 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:25.350663 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:25.350677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.352822 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.352830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:25.357175 (+ 4345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:25.357193 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.359934 (+ 2741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.359943 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:25.360222 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:25.360271 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 12:03:25.361560 (+ 1289us) client_negotiation.cc:240] Negotiation successful
0809 12:03:25.361777 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1926,"mutex_wait_us":70,"thread_start_us":966,"threads_started":1}
I20260809 12:03:25.368018 2547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1f2b42b5da4afa8f1b2a9a2d32e7cc" candidate_uuid: "103d12fe7c3b40f281e9bb71b9630b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff2488bac874be5962e011227e7b515" is_pre_election: true
I20260809 12:03:25.368726 2547 raft_consensus.cc:2454] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d12fe7c3b40f281e9bb71b9630b10 in term 0.
I20260809 12:03:25.413053 2856 raft_consensus.cc:504] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.414081 2856 leader_election.cc:288] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [CANDIDATE]: Term 1 election: Requested vote from peers 6ff2488bac874be5962e011227e7b515 (127.0.147.66:32965), 1f3fefa9119c42459b0c0bf7aa1d7584 (127.0.147.67:43651)
I20260809 12:03:25.414806 2547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1f2b42b5da4afa8f1b2a9a2d32e7cc" candidate_uuid: "103d12fe7c3b40f281e9bb71b9630b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff2488bac874be5962e011227e7b515"
I20260809 12:03:25.415134 2547 raft_consensus.cc:3037] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:25.416481 2754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1f2b42b5da4afa8f1b2a9a2d32e7cc" candidate_uuid: "103d12fe7c3b40f281e9bb71b9630b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584"
I20260809 12:03:25.416719 2754 raft_consensus.cc:3037] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:25.419871 2754 raft_consensus.cc:2454] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d12fe7c3b40f281e9bb71b9630b10 in term 1.
I20260809 12:03:25.420085 2547 raft_consensus.cc:2454] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d12fe7c3b40f281e9bb71b9630b10 in term 1.
I20260809 12:03:25.420759 2253 leader_election.cc:302] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d12fe7c3b40f281e9bb71b9630b10, 1f3fefa9119c42459b0c0bf7aa1d7584; no voters:
I20260809 12:03:25.421357 2856 raft_consensus.cc:2781] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:25.422612 2856 raft_consensus.cc:686] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 LEADER]: Becoming Leader. State: Replica: 103d12fe7c3b40f281e9bb71b9630b10, State: Running, Role: LEADER
I20260809 12:03:25.423470 2856 consensus_queue.cc:237] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.428714 2133 catalog_manager.cc:5495] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d12fe7c3b40f281e9bb71b9630b10 (127.0.147.65). New cstate: current_term: 1 leader_uuid: "103d12fe7c3b40f281e9bb71b9630b10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } health_report { overall_health: UNKNOWN } } }
Aug 09 12:03:25 dist-test-slave-sb2z krb5kdc[2069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277002, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:25.537648 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:25.506544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49416 (local address 127.0.147.126:33527)
0809 12:03:25.511466 (+ 4922us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:25.511470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:25.511574 (+ 104us) server_negotiation.cc:407] Connection header received
0809 12:03:25.511749 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:25.511754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:25.511817 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:25.511927 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:25.515372 (+ 3445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.516755 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.520987 (+ 4232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.522320 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.522487 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:25.529374 (+ 6887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:25.529379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:25.529383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:25.529416 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:25.533890 (+ 4474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:25.534718 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:25.534725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:25.534729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:25.534985 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:25.535446 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:25.535451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:25.535455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:25.535870 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:25.536054 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:25.536515 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 12:03:25.536898 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4760,"thread_start_us":4540,"threads_started":1}
I20260809 12:03:25.537801 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:25.506558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:33527 (local address 127.0.0.1:49416)
0809 12:03:25.511245 (+ 4687us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:25.511273 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:25.511571 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:25.512126 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:25.512132 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:25.514123 (+ 1991us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:25.515134 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:25.515152 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.516926 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.516932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:25.520777 (+ 3845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:25.520798 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:25.522679 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:25.522689 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:25.523026 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:25.524393 (+ 1367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:25.524475 (+ 82us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:25.529128 (+ 4653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:25.534077 (+ 4949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:25.534092 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:25.534102 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:25.534577 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:25.535124 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:25.535130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:25.535134 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:25.535320 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:25.536018 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:25.536028 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:25.536235 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 12:03:25.536410 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 12:03:25.537063 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4378,"mutex_wait_us":265,"thread_start_us":4017,"threads_started":1}
I20260809 12:03:25.544487 2133 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:49416:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 12:03:25.545678 2133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:25.553514 2319 tablet_service.cc:1467] Processing CreateTablet for tablet 8d22104d01054a92a9466b0c75553c17 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66f078f359b2493ea710fd471f46a47f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:03:25.553676 2521 tablet_service.cc:1467] Processing CreateTablet for tablet 8d22104d01054a92a9466b0c75553c17 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66f078f359b2493ea710fd471f46a47f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:03:25.553872 2718 tablet_service.cc:1467] Processing CreateTablet for tablet 8d22104d01054a92a9466b0c75553c17 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66f078f359b2493ea710fd471f46a47f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:03:25.554296 2319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d22104d01054a92a9466b0c75553c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:25.554433 2521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d22104d01054a92a9466b0c75553c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:25.554787 2718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d22104d01054a92a9466b0c75553c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:25.571152 2909 tablet_bootstrap.cc:492] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: Bootstrap starting.
I20260809 12:03:25.573645 2908 tablet_bootstrap.cc:492] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: Bootstrap starting.
I20260809 12:03:25.575362 2904 tablet_bootstrap.cc:492] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: Bootstrap starting.
I20260809 12:03:25.577783 2908 tablet_bootstrap.cc:654] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:25.578516 2909 tablet_bootstrap.cc:654] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:25.582754 2909 tablet_bootstrap.cc:492] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: No bootstrap required, opened a new log
I20260809 12:03:25.583071 2909 ts_tablet_manager.cc:1397] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260809 12:03:25.583227 2904 tablet_bootstrap.cc:654] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:25.584700 2908 tablet_bootstrap.cc:492] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: No bootstrap required, opened a new log
I20260809 12:03:25.584709 2909 raft_consensus.cc:348] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.585147 2908 ts_tablet_manager.cc:1397] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260809 12:03:25.585152 2909 raft_consensus.cc:374] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:25.585415 2909 raft_consensus.cc:729] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d12fe7c3b40f281e9bb71b9630b10, State: Initialized, Role: FOLLOWER
I20260809 12:03:25.585878 2909 consensus_queue.cc:260] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.586585 2856 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:25.587069 2856 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:25.587879 2908 raft_consensus.cc:348] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.588402 2908 raft_consensus.cc:374] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:25.588732 2908 raft_consensus.cc:729] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3fefa9119c42459b0c0bf7aa1d7584, State: Initialized, Role: FOLLOWER
I20260809 12:03:25.589195 2908 consensus_queue.cc:260] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.592177 2909 ts_tablet_manager.cc:1428] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: Time spent starting tablet: real 0.009s user 0.001s sys 0.004s
I20260809 12:03:25.595572 2904 tablet_bootstrap.cc:492] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: No bootstrap required, opened a new log
I20260809 12:03:25.595921 2904 ts_tablet_manager.cc:1397] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.010s
I20260809 12:03:25.596279 2908 ts_tablet_manager.cc:1428] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: Time spent starting tablet: real 0.011s user 0.004s sys 0.005s
I20260809 12:03:25.597339 2904 raft_consensus.cc:348] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.597739 2904 raft_consensus.cc:374] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:25.598038 2904 raft_consensus.cc:729] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff2488bac874be5962e011227e7b515, State: Initialized, Role: FOLLOWER
I20260809 12:03:25.598397 2904 consensus_queue.cc:260] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.599159 2915 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:25.599749 2915 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:25.600869 2904 ts_tablet_manager.cc:1428] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260809 12:03:25.601866 2917 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:25.602517 2917 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:25.875885 2917 raft_consensus.cc:482] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:25.876122 2917 raft_consensus.cc:504] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.876881 2917 leader_election.cc:288] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d12fe7c3b40f281e9bb71b9630b10 (127.0.147.65:40583), 1f3fefa9119c42459b0c0bf7aa1d7584 (127.0.147.67:43651)
I20260809 12:03:25.877496 2346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d22104d01054a92a9466b0c75553c17" candidate_uuid: "6ff2488bac874be5962e011227e7b515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d12fe7c3b40f281e9bb71b9630b10" is_pre_election: true
I20260809 12:03:25.877625 2754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d22104d01054a92a9466b0c75553c17" candidate_uuid: "6ff2488bac874be5962e011227e7b515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" is_pre_election: true
I20260809 12:03:25.877811 2346 raft_consensus.cc:2454] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff2488bac874be5962e011227e7b515 in term 0.
I20260809 12:03:25.877871 2754 raft_consensus.cc:2454] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff2488bac874be5962e011227e7b515 in term 0.
I20260809 12:03:25.878347 2448 leader_election.cc:302] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d12fe7c3b40f281e9bb71b9630b10, 6ff2488bac874be5962e011227e7b515; no voters:
I20260809 12:03:25.878680 2917 raft_consensus.cc:2781] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:25.878856 2917 raft_consensus.cc:482] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:25.878980 2917 raft_consensus.cc:3037] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:25.926402 2917 raft_consensus.cc:504] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.927277 2917 leader_election.cc:288] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [CANDIDATE]: Term 1 election: Requested vote from peers 103d12fe7c3b40f281e9bb71b9630b10 (127.0.147.65:40583), 1f3fefa9119c42459b0c0bf7aa1d7584 (127.0.147.67:43651)
I20260809 12:03:25.928009 2346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d22104d01054a92a9466b0c75553c17" candidate_uuid: "6ff2488bac874be5962e011227e7b515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d12fe7c3b40f281e9bb71b9630b10"
I20260809 12:03:25.928107 2754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d22104d01054a92a9466b0c75553c17" candidate_uuid: "6ff2488bac874be5962e011227e7b515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584"
I20260809 12:03:25.928288 2346 raft_consensus.cc:3037] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:25.928463 2754 raft_consensus.cc:3037] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:25.930935 2346 raft_consensus.cc:2454] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff2488bac874be5962e011227e7b515 in term 1.
I20260809 12:03:25.931468 2448 leader_election.cc:302] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d12fe7c3b40f281e9bb71b9630b10, 6ff2488bac874be5962e011227e7b515; no voters:
I20260809 12:03:25.931840 2917 raft_consensus.cc:2781] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:25.932119 2754 raft_consensus.cc:2454] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff2488bac874be5962e011227e7b515 in term 1.
I20260809 12:03:25.932503 2917 raft_consensus.cc:686] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [term 1 LEADER]: Becoming Leader. State: Replica: 6ff2488bac874be5962e011227e7b515, State: Running, Role: LEADER
I20260809 12:03:25.932837 2917 consensus_queue.cc:237] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } }
I20260809 12:03:25.934985 2918 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: TxnStatusTablet state changed. Reason: New leader 6ff2488bac874be5962e011227e7b515. Latest consensus state: current_term: 1 leader_uuid: "6ff2488bac874be5962e011227e7b515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:25.935310 2918 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:03:25.936453 2920 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 12:03:25.936769 2920 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 12:03:25.936945 2134 catalog_manager.cc:5495] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ff2488bac874be5962e011227e7b515 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "6ff2488bac874be5962e011227e7b515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } health_report { overall_health: UNKNOWN } } }
I20260809 12:03:25.942373 2856 consensus_queue.cc:1035] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:25.951802 2856 consensus_queue.cc:1035] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:26.047330 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.039636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:32965 (local address 127.0.0.1:57260)
0809 12:03:26.040184 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:26.040220 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:26.040767 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:26.041291 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:26.041294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:26.041304 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:03:26.041515 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:26.041524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.043359 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.043363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:26.044499 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:26.044512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.045944 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.045949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:26.046082 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.046142 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:03:26.046707 (+ 565us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:03:26.046734 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 12:03:26.046800 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 12:03:26.046913 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":250,"threads_started":1}
I20260809 12:03:26.047451 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.039924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.0.147.66:32965)
0809 12:03:26.040918 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:26.040925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:26.040952 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:03:26.041035 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:26.041040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:26.041130 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:26.041272 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:26.041674 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.043165 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.044643 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.045720 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.045861 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.046425 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:26.046589 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:26.046680 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:26.046850 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 12:03:26.046963 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":607,"threads_started":1}
I20260809 12:03:26.052712 2754 raft_consensus.cc:1261] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 1 FOLLOWER]: Refusing update from remote peer 6ff2488bac874be5962e011227e7b515: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:26.052707 2346 raft_consensus.cc:1261] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 FOLLOWER]: Refusing update from remote peer 6ff2488bac874be5962e011227e7b515: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:26.053552 2917 consensus_queue.cc:1035] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:26.053871 2918 consensus_queue.cc:1035] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:26.057772 2915 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: TxnStatusTablet state changed. Reason: New leader 6ff2488bac874be5962e011227e7b515. Latest consensus state: current_term: 1 leader_uuid: "6ff2488bac874be5962e011227e7b515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:26.058131 2915 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:26.060582 2930 mvcc.cc:205] Tried to move back new op lower bound from 7316590616784158720 to 7316590616304283648. Current Snapshot: MvccSnapshot[applied={T|T < 7316590616784158720}]
I20260809 12:03:26.061257 2927 mvcc.cc:205] Tried to move back new op lower bound from 7316590616784158720 to 7316590616304283648. Current Snapshot: MvccSnapshot[applied={T|T < 7316590616784158720}]
I20260809 12:03:26.063866 2925 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: TxnStatusTablet state changed. Reason: New leader 6ff2488bac874be5962e011227e7b515. Latest consensus state: current_term: 1 leader_uuid: "6ff2488bac874be5962e011227e7b515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:26.064477 2925 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:26.073565 2925 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ff2488bac874be5962e011227e7b515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:26.073939 2925 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 103d12fe7c3b40f281e9bb71b9630b10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:26.075333 2915 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ff2488bac874be5962e011227e7b515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:26.075805 2915 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 1f3fefa9119c42459b0c0bf7aa1d7584: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:26.077025 2918 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ff2488bac874be5962e011227e7b515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:26.077450 2918 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:03:26.078483 2917 tablet_replica.cc:439] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ff2488bac874be5962e011227e7b515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d12fe7c3b40f281e9bb71b9630b10" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40583 } } peers { permanent_uuid: "6ff2488bac874be5962e011227e7b515" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 32965 } } peers { permanent_uuid: "1f3fefa9119c42459b0c0bf7aa1d7584" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43651 } } }
I20260809 12:03:26.078890 2917 tablet_replica.cc:442] T 8d22104d01054a92a9466b0c75553c17 P 6ff2488bac874be5962e011227e7b515: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:03:26.102744 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.088664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57598 (local address 127.0.147.65:40583)
0809 12:03:26.094047 (+ 5383us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:26.094052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:26.094070 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:03:26.094160 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:26.094163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:26.094377 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:26.094673 (+ 296us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:26.095261 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.096793 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.099240 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.099978 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.100136 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.100802 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:26.101780 (+ 978us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:26.102013 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:26.102070 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 12:03:26.102173 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5209,"thread_start_us":5075,"threads_started":1}
I20260809 12:03:26.127765 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.116772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:46445 (local address 127.0.147.66:32965)
0809 12:03:26.117006 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:26.117012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:26.117665 (+ 653us) server_negotiation.cc:407] Connection header received
0809 12:03:26.117894 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:26.117900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:26.118009 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:26.118134 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:26.119316 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.120928 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.125035 (+ 4107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.126050 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.126193 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.126838 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:26.126991 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:26.127088 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:26.127281 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 12:03:26.127365 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 12:03:26.127816 2953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.116248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:32965 (local address 127.0.147.65:46445)
0809 12:03:26.117513 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:26.117530 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:26.117744 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:26.118186 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:26.118192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:26.118217 (+ 25us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:03:26.119027 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:26.119052 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.121159 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.121165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:26.124819 (+ 3654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:26.124847 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.126394 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.126407 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:26.126553 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.126610 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:03:26.127138 (+ 528us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:03:26.127155 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 12:03:26.127233 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 12:03:26.127344 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":773,"threads_started":1}
I20260809 12:03:26.175930 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.167697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:60361 (local address 127.0.147.65:40583)
0809 12:03:26.167902 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:26.167908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:26.169324 (+ 1416us) server_negotiation.cc:407] Connection header received
0809 12:03:26.169555 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:26.169561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:26.169639 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:26.169760 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:26.170729 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.171969 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.173919 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.174403 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.174533 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.175030 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:26.175125 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:26.175244 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:26.175422 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 12:03:26.175509 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:03:26.176064 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.167455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40583 (local address 127.0.147.66:60361)
0809 12:03:26.169225 (+ 1770us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:26.169242 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:26.169414 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:26.169837 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:26.169844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:26.169859 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:03:26.170473 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:26.170487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.172121 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.172127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:26.173768 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:26.173780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.174570 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.174577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:26.174764 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.174817 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:03:26.175300 (+ 483us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:03:26.175309 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 12:03:26.175397 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 12:03:26.175532 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1544,"thread_start_us":1316,"threads_started":1}
I20260809 12:03:26.220533 2134 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:26.220863 2134 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:26.225060 2134 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd8a56a88664496ca4e323fb6ae48b4d: Sending DeleteTablet for 3 replicas of tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc
I20260809 12:03:26.225962 2319 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:26 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:57596
I20260809 12:03:26.226785 2521 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:26 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:57250
I20260809 12:03:26.230728 2718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:26 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:35410
I20260809 12:03:26.237042 2960 tablet_replica.cc:330] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: stopping tablet replica
I20260809 12:03:26.242107 2960 raft_consensus.cc:2229] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:03:26.243101 2960 raft_consensus.cc:2258] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:26.244516 2960 ts_tablet_manager.cc:1905] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:26.252646 2962 tablet_replica.cc:330] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: stopping tablet replica
I20260809 12:03:26.252916 2962 raft_consensus.cc:2229] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:26.253362 2962 raft_consensus.cc:2258] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:26.254841 2962 ts_tablet_manager.cc:1905] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:26.266428 2964 tablet_replica.cc:330] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: stopping tablet replica
I20260809 12:03:26.266779 2964 raft_consensus.cc:2229] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:26.267257 2964 raft_consensus.cc:2258] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:26.268770 2964 ts_tablet_manager.cc:1905] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 12:03:26 dist-test-slave-sb2z krb5kdc[2069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:26.297509 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.250633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57604 (local address 127.0.147.65:40583)
0809 12:03:26.250860 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:26.250867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:26.256528 (+ 5661us) server_negotiation.cc:407] Connection header received
0809 12:03:26.256789 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:26.256798 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:26.256908 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:26.257042 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:26.269934 (+ 12892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.271367 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.273626 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.274421 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.274545 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.286346 (+ 11801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:26.286392 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:26.286414 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:26.286470 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:26.292769 (+ 6299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.294541 (+ 1772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.294551 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.294556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.294814 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.295396 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.295401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.295405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.296038 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:26.296524 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:26.296711 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 12:03:26.296951 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 12:03:26.298528 2413 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57604
I20260809 12:03:26.303170 2960 ts_tablet_manager.cc:1918] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:03:26.303874 2962 ts_tablet_manager.cc:1918] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:03:26.304090 2962 log.cc:1196] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/6f1f2b42b5da4afa8f1b2a9a2d32e7cc
I20260809 12:03:26.304083 2964 ts_tablet_manager.cc:1918] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:03:26.304317 2964 log.cc:1196] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/6f1f2b42b5da4afa8f1b2a9a2d32e7cc
I20260809 12:03:26.304684 2962 ts_tablet_manager.cc:1939] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 6ff2488bac874be5962e011227e7b515: Deleting consensus metadata
I20260809 12:03:26.304960 2964 ts_tablet_manager.cc:1939] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 1f3fefa9119c42459b0c0bf7aa1d7584: Deleting consensus metadata
I20260809 12:03:26.306170 2119 catalog_manager.cc:4841] TS 6ff2488bac874be5962e011227e7b515 (127.0.147.66:32965): tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc (table test-table [id=43885e216a404d67831ed4184a6f97cf]) successfully deleted
I20260809 12:03:26.306742 2117 catalog_manager.cc:4841] TS 1f3fefa9119c42459b0c0bf7aa1d7584 (127.0.147.67:43651): tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc (table test-table [id=43885e216a404d67831ed4184a6f97cf]) successfully deleted
I20260809 12:03:26.307088 2960 log.cc:1196] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/6f1f2b42b5da4afa8f1b2a9a2d32e7cc
I20260809 12:03:26.307878 2960 ts_tablet_manager.cc:1939] T 6f1f2b42b5da4afa8f1b2a9a2d32e7cc P 103d12fe7c3b40f281e9bb71b9630b10: Deleting consensus metadata
I20260809 12:03:26.309269 2117 catalog_manager.cc:4841] TS 103d12fe7c3b40f281e9bb71b9630b10 (127.0.147.65:40583): tablet 6f1f2b42b5da4afa8f1b2a9a2d32e7cc (table test-table [id=43885e216a404d67831ed4184a6f97cf]) successfully deleted
I20260809 12:03:26.340941 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.306700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.0.147.126:33527)
0809 12:03:26.310266 (+ 3566us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:26.310274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:26.319618 (+ 9344us) server_negotiation.cc:407] Connection header received
0809 12:03:26.319693 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:26.319699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:26.319809 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:26.319928 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:26.322875 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.324461 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.326533 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.327270 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.327385 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.331764 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:26.331772 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:26.331776 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:26.331816 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:26.336555 (+ 4739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.337667 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.337673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.337677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.337884 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.338487 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.338493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.338497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.338928 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:26.339291 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:26.340040 (+ 749us) server_negotiation.cc:299] Negotiation successful
0809 12:03:26.340327 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3334,"thread_start_us":3066,"threads_started":1}
W20260809 12:03:26.343084 2134 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49422
I20260809 12:03:26.347998 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 2228
I20260809 12:03:26.371696 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.350972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57612 (local address 127.0.147.65:40583)
0809 12:03:26.351144 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:26.351153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:26.356582 (+ 5429us) server_negotiation.cc:407] Connection header received
0809 12:03:26.356785 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:26.356791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:26.356869 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:26.356987 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:26.358469 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.359212 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.360685 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.361237 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.361381 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.364291 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:26.364296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:26.364299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:26.364330 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:26.368489 (+ 4159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.369132 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.369138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.369141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.369322 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.369800 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.369805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.369809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.370092 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:26.370543 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:26.370748 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 12:03:26.371120 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 12:03:26.372473 2413 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57612
W20260809 12:03:26.373049 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 2228: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:03:26.373232 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 2228
I20260809 12:03:26.400624 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 2425
Aug 09 12:03:26 dist-test-slave-sb2z krb5kdc[2069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:03:26.418243 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.401496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57264 (local address 127.0.147.66:32965)
0809 12:03:26.401679 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:26.401686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:26.401710 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:03:26.401924 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:26.401928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:26.402026 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:26.402147 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:26.403927 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.404747 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.406289 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.406745 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.406907 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.410971 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:26.410990 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:26.411011 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:26.411059 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:26.415182 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.415896 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.415907 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.415917 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.416099 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.416516 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.416522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.416526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.417042 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:26.417372 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:26.417576 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 12:03:26.417822 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 12:03:26.419111 2614 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57264
W20260809 12:03:26.419881 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 2425: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:03:26.420069 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 2425
I20260809 12:03:26.454308 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 2628
Aug 09 12:03:26 dist-test-slave-sb2z krb5kdc[2069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:03:26.469569 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:26.455167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35416 (local address 127.0.147.67:43651)
0809 12:03:26.455785 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:26.455790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:26.455808 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:03:26.455863 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:26.455867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:26.455926 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:26.456021 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:26.457259 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.458034 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.459213 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:26.459749 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:26.459851 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:26.463937 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:26.463952 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:26.463968 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:26.464008 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:26.466828 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.467501 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.467519 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.467653 (+ 134us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.467780 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:26.468090 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:26.468093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:26.468095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:26.468432 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:26.468661 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:26.468985 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 12:03:26.469190 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":297,"threads_started":1}
W20260809 12:03:26.470305 2812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35416
W20260809 12:03:26.470973 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 2628: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:03:26.471101 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 2628
I20260809 12:03:26.493826 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 2097
I20260809 12:03:26.494910 2191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:26.612980 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 2097
2026-08-09T12:03:26Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5916 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:03:26 dist-test-slave-sb2z krb5kdc[2977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:03:26 dist-test-slave-sb2z krb5kdc[2977](info): set up 2 sockets
Aug 09 12:03:26 dist-test-slave-sb2z krb5kdc[2977](info): commencing operation
krb5kdc: starting...
W20260809 12:03:27.936566 589 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:03:28 dist-test-slave-sb2z krb5kdc[2977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:03:28.163257 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:35755
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:33433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:35755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:28.491029 2997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:28.491364 2997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:28.491600 2997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:28.498692 2997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:28.498833 2997 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 12:03:28.498932 2997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:28.499060 2997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:28.499158 2997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:28.499249 2997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:28.509117 2997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:33433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:35755
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:35755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:28.510368 2997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:28.511761 2997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:28.523763 3004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:28.523838 3003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:28.523816 3007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:28.524770 2997 server_base.cc:1034] running on GCE node
I20260809 12:03:28.525549 2997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:28.526934 2997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:28.528148 2997 hybrid_clock.cc:648] HybridClock initialized: now 1786277008528077 us; error 54 us; skew 500 ppm
I20260809 12:03:28.528475 2997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:28 dist-test-slave-sb2z krb5kdc[2977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277008, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:28.531752 2997 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:03:28.533155 2997 webserver.cc:460] Webserver started at http://127.0.147.126:41191/ using document root <none> and password file <none>
I20260809 12:03:28.533799 2997 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:28.533885 2997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:28.534157 2997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:28.537163 2997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "796728bb827b425e9b6038ca93cb0de7"
format_stamp: "Formatted at 2026-08-09 12:03:28 on dist-test-slave-sb2z"
server_key: "ba7a6204b5b8a9fc17d5011725c9aeb9"
server_key_iv: "c5ea0d88c09a7d97db1274a6d9d29451"
server_key_version: "encryptionkey@0"
I20260809 12:03:28.537894 2997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "796728bb827b425e9b6038ca93cb0de7"
format_stamp: "Formatted at 2026-08-09 12:03:28 on dist-test-slave-sb2z"
server_key: "ba7a6204b5b8a9fc17d5011725c9aeb9"
server_key_iv: "c5ea0d88c09a7d97db1274a6d9d29451"
server_key_version: "encryptionkey@0"
I20260809 12:03:28.543272 2997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:03:28.546675 3016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:28.548081 2997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:28.548264 2997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "796728bb827b425e9b6038ca93cb0de7"
format_stamp: "Formatted at 2026-08-09 12:03:28 on dist-test-slave-sb2z"
server_key: "ba7a6204b5b8a9fc17d5011725c9aeb9"
server_key_iv: "c5ea0d88c09a7d97db1274a6d9d29451"
server_key_version: "encryptionkey@0"
I20260809 12:03:28.548475 2997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:28.564838 2997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:28.568812 2997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:28.569101 2997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:28.591320 2997 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:35755
I20260809 12:03:28.591348 3092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:35755 every 8 connection(s)
I20260809 12:03:28.592724 2997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:28.596386 3094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:28.602887 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 2997
I20260809 12:03:28.603114 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:28.603813 589 external_mini_cluster.cc:1428] Setting key 9050482e9f9283d63dff2b3d0fe38493
I20260809 12:03:28.607139 3094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7: Bootstrap starting.
I20260809 12:03:28.611275 3094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:28.612684 3094 log.cc:824] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:28.615588 3094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7: No bootstrap required, opened a new log
Aug 09 12:03:28 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:28.624894 3094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796728bb827b425e9b6038ca93cb0de7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35755 } }
I20260809 12:03:28.625357 3094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:28.625465 3094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 796728bb827b425e9b6038ca93cb0de7, State: Initialized, Role: FOLLOWER
I20260809 12:03:28.626151 3094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796728bb827b425e9b6038ca93cb0de7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35755 } }
I20260809 12:03:28.626444 3094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:28.626616 3094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:28.626820 3094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:28.629096 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:28.605842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.0.147.126:35755)
0809 12:03:28.606699 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:28.606717 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:28.606771 (+ 54us) server_negotiation.cc:407] Connection header received
0809 12:03:28.609293 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:28.609328 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:28.609812 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:28.610224 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:28.611638 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:28.613594 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:28.614912 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:28.615961 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:28.616177 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:28.620128 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:28.620139 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:28.620157 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:28.620211 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:28.624358 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:28.625022 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:28.625030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:28.625038 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:28.625241 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:28.625757 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:28.625762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:28.625765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:28.626461 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:28.626781 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:28.627246 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 12:03:28.627713 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":371,"threads_started":1}
I20260809 12:03:28.629980 3094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796728bb827b425e9b6038ca93cb0de7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35755 } }
I20260809 12:03:28.630491 3094 leader_election.cc:302] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 796728bb827b425e9b6038ca93cb0de7; no voters:
I20260809 12:03:28.631080 3094 leader_election.cc:288] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:28.631238 3100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:28.631959 3100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [term 1 LEADER]: Becoming Leader. State: Replica: 796728bb827b425e9b6038ca93cb0de7, State: Running, Role: LEADER
I20260809 12:03:28.632531 3100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796728bb827b425e9b6038ca93cb0de7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35755 } }
I20260809 12:03:28.634057 3094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:28.635501 3101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 796728bb827b425e9b6038ca93cb0de7. Latest consensus state: current_term: 1 leader_uuid: "796728bb827b425e9b6038ca93cb0de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796728bb827b425e9b6038ca93cb0de7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35755 } } }
I20260809 12:03:28.635936 3101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:28.635476 3102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "796728bb827b425e9b6038ca93cb0de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796728bb827b425e9b6038ca93cb0de7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35755 } } }
I20260809 12:03:28.636936 3102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:28.638990 3105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:28.644706 3105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:28.653020 3105 catalog_manager.cc:1347] Generated new cluster ID: d1badb0a4df94b21924ad244163f66b6
I20260809 12:03:28.653134 3105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:28.662976 3105 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:28.664793 3105 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:28.676370 3105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7: Generated new TSK 0
I20260809 12:03:28.677397 3105 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:03:28.736809 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:35755
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:33433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:29.076452 3128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:29.076779 3128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:29.076975 3128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:29.084214 3128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:29.084375 3128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:29.084530 3128 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 12:03:29.084625 3128 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:03:29.084767 3128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:03:29.094524 3128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:33433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.147.126:35755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:29.095912 3128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:29.097298 3128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:29.110178 3139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:29.110363 3136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:29.110337 3137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:29.111593 3128 server_base.cc:1034] running on GCE node
I20260809 12:03:29.112228 3128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:29.113159 3128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:29.114403 3128 hybrid_clock.cc:648] HybridClock initialized: now 1786277009114358 us; error 56 us; skew 500 ppm
I20260809 12:03:29.114724 3128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:29 dist-test-slave-sb2z krb5kdc[2977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277009, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:29.117903 3128 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:03:29.119338 3128 webserver.cc:460] Webserver started at http://127.0.147.65:36245/ using document root <none> and password file <none>
I20260809 12:03:29.120064 3128 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:29.120218 3128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:29.120538 3128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:29.123404 3128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "ab63f560bbb9499b9f8156a7a36b9923"
format_stamp: "Formatted at 2026-08-09 12:03:29 on dist-test-slave-sb2z"
server_key: "6d2e8383419a285f199376c4878cd3f8"
server_key_iv: "7e3ef63d368b3b2b5dcfe5e6bb5b253b"
server_key_version: "encryptionkey@0"
I20260809 12:03:29.124261 3128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "ab63f560bbb9499b9f8156a7a36b9923"
format_stamp: "Formatted at 2026-08-09 12:03:29 on dist-test-slave-sb2z"
server_key: "6d2e8383419a285f199376c4878cd3f8"
server_key_iv: "7e3ef63d368b3b2b5dcfe5e6bb5b253b"
server_key_version: "encryptionkey@0"
I20260809 12:03:29.129429 3128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:03:29.132953 3149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:29.134470 3128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:03:29.134603 3128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "ab63f560bbb9499b9f8156a7a36b9923"
format_stamp: "Formatted at 2026-08-09 12:03:29 on dist-test-slave-sb2z"
server_key: "6d2e8383419a285f199376c4878cd3f8"
server_key_iv: "7e3ef63d368b3b2b5dcfe5e6bb5b253b"
server_key_version: "encryptionkey@0"
I20260809 12:03:29.134835 3128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:29.156646 3128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:29.160940 3128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:29.161247 3128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:29.164203 3128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:29.164366 3128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:29.164502 3128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:29.164603 3128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:03:29 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277009, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:29.186882 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:29.164531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:47255 (local address 127.0.147.126:35755)
0809 12:03:29.165041 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:29.165048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:29.166273 (+ 1225us) server_negotiation.cc:407] Connection header received
0809 12:03:29.168414 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:29.168421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:29.168517 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:29.168654 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:29.170717 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.172088 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.174723 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.175216 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.175370 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:29.180853 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:29.180858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:29.180861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:29.180894 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:29.183523 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:29.184552 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:29.184568 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:29.184571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:29.184700 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:29.185147 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:29.185153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:29.185157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:29.185434 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:29.185602 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:29.186278 (+ 676us) server_negotiation.cc:299] Negotiation successful
0809 12:03:29.186507 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":263,"threads_started":1}
I20260809 12:03:29.188160 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:29.165178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35755 (local address 127.0.147.65:47255)
0809 12:03:29.166080 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:29.166132 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:29.168136 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:29.168885 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:29.168893 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:29.169655 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:29.170436 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:29.170459 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.172287 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.172292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:29.174565 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:29.174577 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.175710 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.175722 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:29.176118 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:29.177041 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:29.177086 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:29.180630 (+ 3544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:29.183725 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:29.183733 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:29.183749 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:29.184423 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:29.184815 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:29.184821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:29.184825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:29.185024 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:29.185566 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:29.185573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:29.185924 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 12:03:29.186196 (+ 272us) client_negotiation.cc:240] Negotiation successful
0809 12:03:29.186707 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":371,"threads_started":1}
I20260809 12:03:29.211611 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:29.194202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35755 (local address 127.0.147.65:44129)
0809 12:03:29.194849 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:29.194864 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:29.195023 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:29.195342 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:29.195346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:29.195911 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:29.196421 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:29.196436 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.197743 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.197748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:29.199480 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:29.199500 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.200759 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.200768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:29.201001 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:29.201884 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:29.201914 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:29.203316 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:29.207245 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:29.207251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:29.207258 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:29.208033 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:29.209345 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:29.209350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:29.209354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:29.209504 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:29.210066 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:29.210071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:29.210333 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 12:03:29.210509 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 12:03:29.210889 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":356,"threads_started":1}
I20260809 12:03:29.212411 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:29.194097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:44129 (local address 127.0.147.126:35755)
0809 12:03:29.194255 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:29.194260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:29.194956 (+ 696us) server_negotiation.cc:407] Connection header received
0809 12:03:29.195147 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:29.195154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:29.195217 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:29.195319 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:29.196563 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.197596 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.199662 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.200317 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.200468 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:29.203556 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:29.203564 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:29.203569 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:29.203607 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:29.207050 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:29.209036 (+ 1986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:29.209050 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:29.209053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:29.209210 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:29.209636 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:29.209641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:29.209644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:29.209933 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:29.210097 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:29.211672 (+ 1575us) server_negotiation.cc:299] Negotiation successful
0809 12:03:29.211973 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:03:29.221365 3128 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:46763
I20260809 12:03:29.221424 3334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:46763 every 8 connection(s)
I20260809 12:03:29.222611 3128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:03:29.224690 3336 heartbeater.cc:347] Connected to a master server at 127.0.147.126:35755
I20260809 12:03:29.224943 3336 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:29.225610 3336 heartbeater.cc:511] Master 127.0.147.126:35755 requested a full tablet report, sending...
I20260809 12:03:29.227470 3037 ts_manager.cc:194] Registered new tserver with Master: ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763)
I20260809 12:03:29.228937 3037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:47255
I20260809 12:03:29.228993 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 3128
I20260809 12:03:29.229195 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:03:29.229936 589 external_mini_cluster.cc:1428] Setting key 4704a9a96bb0027533b95ceeada6f9d2
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:03:29.276525 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:35755
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:33433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:29.615494 3342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:29.615833 3342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:29.615939 3342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:29.623040 3342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:29.623144 3342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:29.623231 3342 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 12:03:29.623286 3342 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:03:29.623416 3342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:03:29.633112 3342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:33433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.147.126:35755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:29.634400 3342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:29.635924 3342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:29.649309 3347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:29.649340 3351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:29.649391 3348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:29.650141 3342 server_base.cc:1034] running on GCE node
I20260809 12:03:29.650684 3342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:29.651479 3342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:29.652727 3342 hybrid_clock.cc:648] HybridClock initialized: now 1786277009652685 us; error 43 us; skew 500 ppm
I20260809 12:03:29.653074 3342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:29 dist-test-slave-sb2z krb5kdc[2977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277009, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:29.656205 3342 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:03:29.657573 3342 webserver.cc:460] Webserver started at http://127.0.147.66:44019/ using document root <none> and password file <none>
I20260809 12:03:29.658237 3342 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:29.658393 3342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:29.658730 3342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:29.661613 3342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "bc59be94d0d44819a079ce07df923fc0"
format_stamp: "Formatted at 2026-08-09 12:03:29 on dist-test-slave-sb2z"
server_key: "4d31494be606ec96cce63df2c0f91898"
server_key_iv: "dfa5c10f7254c27ca7504408ab1b32eb"
server_key_version: "encryptionkey@0"
I20260809 12:03:29.662451 3342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "bc59be94d0d44819a079ce07df923fc0"
format_stamp: "Formatted at 2026-08-09 12:03:29 on dist-test-slave-sb2z"
server_key: "4d31494be606ec96cce63df2c0f91898"
server_key_iv: "dfa5c10f7254c27ca7504408ab1b32eb"
server_key_version: "encryptionkey@0"
I20260809 12:03:29.667618 3342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:03:29.671095 3359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:29.672429 3342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:03:29.672612 3342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "bc59be94d0d44819a079ce07df923fc0"
format_stamp: "Formatted at 2026-08-09 12:03:29 on dist-test-slave-sb2z"
server_key: "4d31494be606ec96cce63df2c0f91898"
server_key_iv: "dfa5c10f7254c27ca7504408ab1b32eb"
server_key_version: "encryptionkey@0"
I20260809 12:03:29.672833 3342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:29.693562 3342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:29.697463 3342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:29.697760 3342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:29.700454 3342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:29.700600 3342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:29.700726 3342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:29.700812 3342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:03:29 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277009, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:29.724498 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:29.700785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:56247 (local address 127.0.147.126:35755)
0809 12:03:29.701006 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:29.701014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:29.702364 (+ 1350us) server_negotiation.cc:407] Connection header received
0809 12:03:29.704455 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:29.704461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:29.704553 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:29.704657 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:29.706648 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.707682 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.711363 (+ 3681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.712102 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.712251 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:29.717495 (+ 5244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:29.717504 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:29.717511 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:29.717562 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:29.720815 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:29.721905 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:29.721908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:29.721911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:29.722025 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:29.722586 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:29.722589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:29.722591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:29.722846 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:29.722956 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:29.724020 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0809 12:03:29.724214 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:03:29.726560 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:29.701319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35755 (local address 127.0.147.66:56247)
0809 12:03:29.702163 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:29.702209 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:29.704162 (+ 1953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:29.704879 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:29.704890 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:29.705573 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:29.706436 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:29.706457 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.707920 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.707929 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:29.711171 (+ 3242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:29.711193 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.712523 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.712529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:29.712945 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:29.713735 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:29.713808 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:29.717259 (+ 3451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:29.720968 (+ 3709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:29.720980 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:29.721000 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:29.721772 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:29.722155 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:29.722162 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:29.722169 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:29.722458 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:29.722988 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:29.723000 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:29.723544 (+ 544us) client_negotiation.cc:769] Sending connection context
0809 12:03:29.723927 (+ 383us) client_negotiation.cc:240] Negotiation successful
0809 12:03:29.724609 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":368,"threads_started":1}
I20260809 12:03:29.749470 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:29.733224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:54557 (local address 127.0.147.126:35755)
0809 12:03:29.733421 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:29.733429 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:29.734282 (+ 853us) server_negotiation.cc:407] Connection header received
0809 12:03:29.734568 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:29.734576 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:29.734664 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:29.734768 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:29.736303 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.737341 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.739779 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.740442 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.740601 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:29.742905 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:29.742914 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:29.742921 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:29.742968 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:29.746272 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:29.747170 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:29.747178 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:29.747183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:29.747380 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:29.747840 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:29.747847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:29.747852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:29.748338 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:29.748505 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:29.748917 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 12:03:29.749136 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:03:29.749675 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:29.733341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35755 (local address 127.0.147.66:54557)
0809 12:03:29.734150 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:29.734174 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:29.734341 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:29.734905 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:29.734909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:29.735709 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:29.736155 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:29.736165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.737479 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.737484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:29.739611 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:29.739629 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:29.740646 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:29.740651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:29.740768 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:29.741467 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:29.741496 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:29.742693 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:29.746416 (+ 3723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:29.746425 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:29.746428 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:29.747044 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:29.747557 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:29.747562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:29.747566 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:29.747723 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:29.748469 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:29.748474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:29.748655 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 12:03:29.748814 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 12:03:29.749203 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":432,"threads_started":1}
I20260809 12:03:29.752796 3342 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:36841
I20260809 12:03:29.753289 3535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:36841 every 8 connection(s)
I20260809 12:03:29.754487 3342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:03:29.756582 3536 heartbeater.cc:347] Connected to a master server at 127.0.147.126:35755
I20260809 12:03:29.756832 3536 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:29.756978 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 3342
I20260809 12:03:29.757169 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:03:29.757622 3536 heartbeater.cc:511] Master 127.0.147.126:35755 requested a full tablet report, sending...
I20260809 12:03:29.757772 589 external_mini_cluster.cc:1428] Setting key 671b6361cc2cc6bce6cc17d8ead332b2
I20260809 12:03:29.759186 3036 ts_manager.cc:194] Registered new tserver with Master: bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841)
I20260809 12:03:29.760210 3036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:56247
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:03:29.806792 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:35755
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:33433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:30.138298 3541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:30.138583 3541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:30.138796 3541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:30.146602 3541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:30.146739 3541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:30.146891 3541 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 12:03:30.146989 3541 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:03:30.147126 3541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:03:30.156961 3541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:33433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.147.126:35755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:30.158298 3541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:30.159679 3541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:30.179679 3551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:30.179747 3552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:30.180397 3555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:30.181180 3541 server_base.cc:1034] running on GCE node
I20260809 12:03:30.181849 3541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:30.182804 3541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:30.184053 3541 hybrid_clock.cc:648] HybridClock initialized: now 1786277010184004 us; error 57 us; skew 500 ppm
I20260809 12:03:30.184388 3541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:30 dist-test-slave-sb2z krb5kdc[2977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277010, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:30.187664 3541 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:03:30.189173 3541 webserver.cc:460] Webserver started at http://127.0.147.67:45527/ using document root <none> and password file <none>
I20260809 12:03:30.189873 3541 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:30.190043 3541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:30.190382 3541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:30.193387 3541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "28ca4b937a93434eb351ab6abd64edb3"
format_stamp: "Formatted at 2026-08-09 12:03:30 on dist-test-slave-sb2z"
server_key: "7699c6d26f3569947bd5b552daa7bd59"
server_key_iv: "8260a2bb2ae65e3b634928c3da691f64"
server_key_version: "encryptionkey@0"
I20260809 12:03:30.194216 3541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "28ca4b937a93434eb351ab6abd64edb3"
format_stamp: "Formatted at 2026-08-09 12:03:30 on dist-test-slave-sb2z"
server_key: "7699c6d26f3569947bd5b552daa7bd59"
server_key_iv: "8260a2bb2ae65e3b634928c3da691f64"
server_key_version: "encryptionkey@0"
I20260809 12:03:30.199473 3541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:03:30.203161 3564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:30.204528 3541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:30.204712 3541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "28ca4b937a93434eb351ab6abd64edb3"
format_stamp: "Formatted at 2026-08-09 12:03:30 on dist-test-slave-sb2z"
server_key: "7699c6d26f3569947bd5b552daa7bd59"
server_key_iv: "8260a2bb2ae65e3b634928c3da691f64"
server_key_version: "encryptionkey@0"
I20260809 12:03:30.204972 3541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:30.222518 3541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:30.226722 3541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:30.227029 3541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:30.230058 3541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:30.230190 3541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:30.230307 3541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:30.230433 3541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:30.232087 3336 heartbeater.cc:503] Master 127.0.147.126:35755 was elected leader, sending a full tablet report...
Aug 09 12:03:30 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277010, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:30.254905 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.230322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:35821 (local address 127.0.147.126:35755)
0809 12:03:30.230514 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.230520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.232114 (+ 1594us) server_negotiation.cc:407] Connection header received
0809 12:03:30.234658 (+ 2544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.234665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.234750 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.234833 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:30.237112 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.238548 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.240822 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.241638 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.241803 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.247136 (+ 5333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:30.247145 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:30.247153 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:30.247206 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:30.250775 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:30.251475 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:30.251481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:30.251484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:30.251678 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:30.252285 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:30.252291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:30.252294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:30.252645 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:30.252823 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.254055 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.254388 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 12:03:30.255954 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.230869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35755 (local address 127.0.147.67:35821)
0809 12:03:30.231891 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.231944 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.234296 (+ 2352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.235275 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.235282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.236034 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:30.236897 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.236932 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.238760 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.238770 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.240667 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.240687 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.242167 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.242177 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.242586 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.243429 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:30.243470 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:30.246855 (+ 3385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:30.250918 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:30.250929 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:30.250945 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:30.251344 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:30.251802 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:30.251811 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:30.251813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:30.252118 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:30.252806 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:30.252814 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:30.253409 (+ 595us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.253893 (+ 484us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.254506 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":388,"threads_started":1}
I20260809 12:03:30.280620 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.263516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:46445 (local address 127.0.147.126:35755)
0809 12:03:30.263722 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.263730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.264395 (+ 665us) server_negotiation.cc:407] Connection header received
0809 12:03:30.264525 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.264533 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.264650 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.264775 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:30.265991 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.267225 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.270041 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.270523 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.270669 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.272768 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:30.272778 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:30.272786 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:30.272836 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:30.277093 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:30.278131 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:30.278134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:30.278137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:30.278253 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:30.278733 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:30.278740 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:30.278743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:30.279109 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:30.279287 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.279944 (+ 657us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.280213 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:03:30.280822 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.263517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35755 (local address 127.0.147.67:46445)
0809 12:03:30.264267 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.264292 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.264414 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.264788 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.264791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.265438 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:30.265787 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.265797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.267458 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.267468 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.269864 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.269883 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.270725 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.270733 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.270915 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.271508 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:30.271579 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:30.272527 (+ 948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:30.277270 (+ 4743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:30.277277 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:30.277282 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:30.277950 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:30.278383 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:30.278391 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:30.278395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:30.278577 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:30.279261 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:30.279267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:30.279601 (+ 334us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.279785 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.280284 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":404,"threads_started":1}
I20260809 12:03:30.289647 3541 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:45035
I20260809 12:03:30.289746 3744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:45035 every 8 connection(s)
I20260809 12:03:30.290963 3541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:03:30.292972 3745 heartbeater.cc:347] Connected to a master server at 127.0.147.126:35755
I20260809 12:03:30.293200 3745 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:30.293831 3745 heartbeater.cc:511] Master 127.0.147.126:35755 requested a full tablet report, sending...
I20260809 12:03:30.295158 3036 ts_manager.cc:194] Registered new tserver with Master: 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035)
I20260809 12:03:30.295708 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 3541
I20260809 12:03:30.295878 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:03:30.296319 3036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:35821
I20260809 12:03:30.296428 589 external_mini_cluster.cc:1428] Setting key 5cb3ecf8451f43be51ff9f78f08d9773
I20260809 12:03:30.299374 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:03:30 dist-test-slave-sb2z krb5kdc[2977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:03:30 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:30.355208 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.341183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.0.147.126:35755)
0809 12:03:30.341357 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.341362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.341653 (+ 291us) server_negotiation.cc:407] Connection header received
0809 12:03:30.341801 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.341804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.341871 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.341960 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:30.343469 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.344284 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.345474 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.345861 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.345951 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.349695 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:30.349700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:30.349703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:30.349737 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:30.352502 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:30.353149 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:30.353159 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:30.353161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:30.353301 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:30.353716 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:30.353721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:30.353724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:30.353990 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:30.354260 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.354590 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.354826 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:03:30.359265 3036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:03:30.366300 3036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:30.393154 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.378610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46763 (local address 127.0.0.1:48064)
0809 12:03:30.380381 (+ 1771us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.380451 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.380755 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.382464 (+ 1709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.382474 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.382541 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:30.383114 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.383131 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.385880 (+ 2749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.385888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.389052 (+ 3164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.389078 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.391841 (+ 2763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.391851 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.392137 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.392215 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.392364 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.392621 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1410,"thread_start_us":1259,"threads_started":1}
I20260809 12:03:30.395176 3772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.379679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.0.147.65:46763)
0809 12:03:30.381812 (+ 2133us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.381820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.381850 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:03:30.381960 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.381966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.382218 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.383400 (+ 1182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:30.383683 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.385681 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.389387 (+ 3706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.391503 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.394108 (+ 2605us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.394423 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.394535 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.394759 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1980,"thread_start_us":1920,"threads_started":1}
I20260809 12:03:30.395869 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.378803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36841 (local address 127.0.0.1:32820)
0809 12:03:30.381764 (+ 2961us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.381789 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.381906 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.384358 (+ 2452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.384366 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.384401 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:30.384899 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.384913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.388523 (+ 3610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.388531 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.392864 (+ 4333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.392884 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.394942 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.394949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.395125 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.395157 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.395372 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.395495 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2833,"thread_start_us":2684,"threads_started":1}
I20260809 12:03:30.396456 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.379571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.0.147.66:36841)
0809 12:03:30.381397 (+ 1826us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.381408 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.381903 (+ 495us) server_negotiation.cc:407] Connection header received
0809 12:03:30.382148 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.382160 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.382447 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.382765 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:30.385132 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.388328 (+ 3196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.393105 (+ 4777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.394752 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.394886 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.395092 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.395661 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.395823 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1639,"thread_start_us":1514,"threads_started":1}
I20260809 12:03:30.399463 3434 tablet_service.cc:1467] Processing CreateTablet for tablet e7c08a6ae7b946ee9f0ad8803f0d24df (DEFAULT_TABLE table=test-table [id=dffd60cc371946739f5b021b74fe304b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:30.399482 3225 tablet_service.cc:1467] Processing CreateTablet for tablet e7c08a6ae7b946ee9f0ad8803f0d24df (DEFAULT_TABLE table=test-table [id=dffd60cc371946739f5b021b74fe304b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:30.400820 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7c08a6ae7b946ee9f0ad8803f0d24df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:30.400884 3434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7c08a6ae7b946ee9f0ad8803f0d24df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:30.404719 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.381892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45035 (local address 127.0.0.1:40736)
0809 12:03:30.393775 (+ 11883us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.393796 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.393908 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.398286 (+ 4378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.398291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.398316 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:30.398458 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.398468 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.400358 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.400364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.402178 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.402192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.403971 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.403977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.404178 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.404210 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.404312 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.404443 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11761,"thread_start_us":12736,"threads_started":1}
I20260809 12:03:30.405041 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.382327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40736 (local address 127.0.147.67:45035)
0809 12:03:30.397689 (+ 15362us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.397698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.397732 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:03:30.397827 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.397834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.398069 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.398278 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:30.398617 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.400202 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.402361 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.403801 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.403930 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.404155 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.404364 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.404518 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15177,"thread_start_us":17988,"threads_started":1}
I20260809 12:03:30.408789 3644 tablet_service.cc:1467] Processing CreateTablet for tablet e7c08a6ae7b946ee9f0ad8803f0d24df (DEFAULT_TABLE table=test-table [id=dffd60cc371946739f5b021b74fe304b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:30.410357 3644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7c08a6ae7b946ee9f0ad8803f0d24df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:30.413605 3776 tablet_bootstrap.cc:492] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: Bootstrap starting.
I20260809 12:03:30.416290 3777 tablet_bootstrap.cc:492] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: Bootstrap starting.
I20260809 12:03:30.416898 3776 tablet_bootstrap.cc:654] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:30.418352 3776 log.cc:824] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:30.420068 3777 tablet_bootstrap.cc:654] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:30.421509 3777 log.cc:824] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:30.422362 3776 tablet_bootstrap.cc:492] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: No bootstrap required, opened a new log
I20260809 12:03:30.422632 3776 ts_tablet_manager.cc:1397] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260809 12:03:30.424564 3777 tablet_bootstrap.cc:492] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: No bootstrap required, opened a new log
I20260809 12:03:30.424824 3777 ts_tablet_manager.cc:1397] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 12:03:30.424857 3781 tablet_bootstrap.cc:492] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: Bootstrap starting.
I20260809 12:03:30.428386 3781 tablet_bootstrap.cc:654] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:30.429600 3781 log.cc:824] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:30.432977 3777 raft_consensus.cc:348] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.433323 3777 raft_consensus.cc:374] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:30.433437 3777 raft_consensus.cc:729] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab63f560bbb9499b9f8156a7a36b9923, State: Initialized, Role: FOLLOWER
I20260809 12:03:30.433183 3776 raft_consensus.cc:348] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.433597 3776 raft_consensus.cc:374] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:30.433710 3776 raft_consensus.cc:729] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc59be94d0d44819a079ce07df923fc0, State: Initialized, Role: FOLLOWER
I20260809 12:03:30.434187 3781 tablet_bootstrap.cc:492] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: No bootstrap required, opened a new log
I20260809 12:03:30.434485 3781 ts_tablet_manager.cc:1397] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260809 12:03:30.434320 3776 consensus_queue.cc:260] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.434353 3777 consensus_queue.cc:260] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.435575 3536 heartbeater.cc:503] Master 127.0.147.126:35755 was elected leader, sending a full tablet report...
I20260809 12:03:30.436160 3776 ts_tablet_manager.cc:1428] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: Time spent starting tablet: real 0.013s user 0.012s sys 0.003s
I20260809 12:03:30.444875 3781 raft_consensus.cc:348] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.445232 3781 raft_consensus.cc:374] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:30.445343 3781 raft_consensus.cc:729] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ca4b937a93434eb351ab6abd64edb3, State: Initialized, Role: FOLLOWER
I20260809 12:03:30.445909 3781 consensus_queue.cc:260] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.447595 3777 ts_tablet_manager.cc:1428] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: Time spent starting tablet: real 0.023s user 0.018s sys 0.004s
I20260809 12:03:30.448027 3745 heartbeater.cc:503] Master 127.0.147.126:35755 was elected leader, sending a full tablet report...
I20260809 12:03:30.448277 3781 ts_tablet_manager.cc:1428] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
I20260809 12:03:30.466483 3787 raft_consensus.cc:482] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:30.466759 3787 raft_consensus.cc:504] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.468381 3787 leader_election.cc:288] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841), 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035)
W20260809 12:03:30.474933 3337 tablet.cc:2369] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:03:30.476979 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.468771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:49593 (local address 127.0.147.66:36841)
0809 12:03:30.468959 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.468963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.469560 (+ 597us) server_negotiation.cc:407] Connection header received
0809 12:03:30.469824 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.469833 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.469932 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.470058 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:30.470719 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.471821 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.474380 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.475669 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.475823 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.475895 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.476572 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.476729 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:03:30.476974 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.469754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:35901 (local address 127.0.147.67:45035)
0809 12:03:30.469979 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.469985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.470331 (+ 346us) server_negotiation.cc:407] Connection header received
0809 12:03:30.470501 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.470506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.470561 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.470653 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:30.471142 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.472128 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.474436 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.475656 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.475823 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.475895 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.476572 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.476716 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:03:30.477412 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.469530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45035 (local address 127.0.147.65:35901)
0809 12:03:30.470208 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.470229 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.470365 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.470673 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.470680 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.470710 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:30.470988 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.470994 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.472256 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.472261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.474290 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.474303 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.475942 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.475951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.476254 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.476410 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.476505 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.476675 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":129,"thread_start_us":347,"threads_started":1}
I20260809 12:03:30.477458 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.468660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36841 (local address 127.0.147.65:49593)
0809 12:03:30.469420 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.469440 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.469633 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.470088 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.470093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.470121 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:30.470551 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.470562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.471942 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.471945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.474190 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.474205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.475959 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.475964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.476250 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.476410 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.476505 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.476675 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":142,"thread_start_us":470,"threads_started":1}
I20260809 12:03:30.478538 3475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c08a6ae7b946ee9f0ad8803f0d24df" candidate_uuid: "ab63f560bbb9499b9f8156a7a36b9923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59be94d0d44819a079ce07df923fc0" is_pre_election: true
I20260809 12:03:30.478524 3680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c08a6ae7b946ee9f0ad8803f0d24df" candidate_uuid: "ab63f560bbb9499b9f8156a7a36b9923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ca4b937a93434eb351ab6abd64edb3" is_pre_election: true
I20260809 12:03:30.479090 3680 raft_consensus.cc:2454] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab63f560bbb9499b9f8156a7a36b9923 in term 0.
I20260809 12:03:30.479099 3475 raft_consensus.cc:2454] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab63f560bbb9499b9f8156a7a36b9923 in term 0.
I20260809 12:03:30.479789 3152 leader_election.cc:302] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ca4b937a93434eb351ab6abd64edb3, ab63f560bbb9499b9f8156a7a36b9923; no voters:
I20260809 12:03:30.480177 3787 raft_consensus.cc:2781] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:30.480365 3787 raft_consensus.cc:482] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:30.480494 3787 raft_consensus.cc:3037] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:30.482748 3787 raft_consensus.cc:504] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.483325 3787 leader_election.cc:288] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [CANDIDATE]: Term 1 election: Requested vote from peers bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841), 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035)
I20260809 12:03:30.483850 3475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c08a6ae7b946ee9f0ad8803f0d24df" candidate_uuid: "ab63f560bbb9499b9f8156a7a36b9923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59be94d0d44819a079ce07df923fc0"
I20260809 12:03:30.483914 3680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c08a6ae7b946ee9f0ad8803f0d24df" candidate_uuid: "ab63f560bbb9499b9f8156a7a36b9923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ca4b937a93434eb351ab6abd64edb3"
I20260809 12:03:30.484066 3475 raft_consensus.cc:3037] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:30.484185 3680 raft_consensus.cc:3037] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:30.486377 3475 raft_consensus.cc:2454] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab63f560bbb9499b9f8156a7a36b9923 in term 1.
I20260809 12:03:30.486433 3680 raft_consensus.cc:2454] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab63f560bbb9499b9f8156a7a36b9923 in term 1.
I20260809 12:03:30.486804 3152 leader_election.cc:302] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab63f560bbb9499b9f8156a7a36b9923, bc59be94d0d44819a079ce07df923fc0; no voters:
I20260809 12:03:30.487128 3787 raft_consensus.cc:2781] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:30.487687 3787 raft_consensus.cc:686] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 1 LEADER]: Becoming Leader. State: Replica: ab63f560bbb9499b9f8156a7a36b9923, State: Running, Role: LEADER
I20260809 12:03:30.488209 3787 consensus_queue.cc:237] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.492970 3036 catalog_manager.cc:5495] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65). New cstate: current_term: 1 leader_uuid: "ab63f560bbb9499b9f8156a7a36b9923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } health_report { overall_health: UNKNOWN } } }
W20260809 12:03:30.506559 3538 tablet.cc:2369] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 12:03:30 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277008, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
W20260809 12:03:30.542690 3746 tablet.cc:2369] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:03:30.544375 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.530248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35755 (local address 127.0.0.1:52376)
0809 12:03:30.530835 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.530850 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.531030 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.531367 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.531371 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.532136 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:30.532418 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.532427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.533489 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.533494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.534615 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.534628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.535397 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.535404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.535628 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.536256 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:30.536277 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:30.538672 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:30.541671 (+ 2999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:30.541682 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:30.541687 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:30.542174 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:30.542660 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:30.542667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:30.542686 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:30.542839 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:30.543469 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:30.543475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:30.543706 (+ 231us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.543842 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.544051 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":333,"threads_started":1}
I20260809 12:03:30.544394 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.530199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52376 (local address 127.0.147.126:35755)
0809 12:03:30.530370 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.530375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.530946 (+ 571us) server_negotiation.cc:407] Connection header received
0809 12:03:30.531159 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.531168 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.531236 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.531342 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:30.532589 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.533348 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.534778 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.535209 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.535354 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.538886 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:30.538898 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:30.538902 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:30.538932 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:30.541548 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:30.542287 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:30.542291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:30.542293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:30.542495 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:30.542965 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:30.542969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:30.542972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:30.543343 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:30.543516 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.543963 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.544154 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":62}
I20260809 12:03:30.549220 3036 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:52376:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 12:03:30.550141 3036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:30.556885 3434 tablet_service.cc:1467] Processing CreateTablet for tablet 744fc6cf7a0f4fda9de3e335a1052b84 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3cac7771bab04e42ba9a93ed96cbc282]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:03:30.556984 3644 tablet_service.cc:1467] Processing CreateTablet for tablet 744fc6cf7a0f4fda9de3e335a1052b84 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3cac7771bab04e42ba9a93ed96cbc282]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:03:30.556883 3225 tablet_service.cc:1467] Processing CreateTablet for tablet 744fc6cf7a0f4fda9de3e335a1052b84 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3cac7771bab04e42ba9a93ed96cbc282]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:03:30.557847 3434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 744fc6cf7a0f4fda9de3e335a1052b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:30.557852 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 744fc6cf7a0f4fda9de3e335a1052b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:30.557853 3644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 744fc6cf7a0f4fda9de3e335a1052b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:30.563885 3776 tablet_bootstrap.cc:492] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: Bootstrap starting.
I20260809 12:03:30.565073 3777 tablet_bootstrap.cc:492] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: Bootstrap starting.
I20260809 12:03:30.565471 3776 tablet_bootstrap.cc:654] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:30.565551 3781 tablet_bootstrap.cc:492] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: Bootstrap starting.
I20260809 12:03:30.566821 3777 tablet_bootstrap.cc:654] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:30.567313 3781 tablet_bootstrap.cc:654] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:30.568113 3776 tablet_bootstrap.cc:492] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: No bootstrap required, opened a new log
I20260809 12:03:30.568286 3776 ts_tablet_manager.cc:1397] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 12:03:30.569185 3776 raft_consensus.cc:348] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.569444 3776 raft_consensus.cc:374] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:30.569564 3776 raft_consensus.cc:729] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc59be94d0d44819a079ce07df923fc0, State: Initialized, Role: FOLLOWER
I20260809 12:03:30.569866 3776 consensus_queue.cc:260] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.570016 3781 tablet_bootstrap.cc:492] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: No bootstrap required, opened a new log
I20260809 12:03:30.570178 3781 ts_tablet_manager.cc:1397] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 12:03:30.570304 3777 tablet_bootstrap.cc:492] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: No bootstrap required, opened a new log
I20260809 12:03:30.570441 3777 ts_tablet_manager.cc:1397] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 12:03:30.570382 3786 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.570787 3786 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:30.570951 3781 raft_consensus.cc:348] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.571187 3781 raft_consensus.cc:374] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:30.571280 3776 ts_tablet_manager.cc:1428] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:30.571296 3781 raft_consensus.cc:729] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ca4b937a93434eb351ab6abd64edb3, State: Initialized, Role: FOLLOWER
I20260809 12:03:30.571200 3777 raft_consensus.cc:348] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.571435 3777 raft_consensus.cc:374] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:30.571548 3777 raft_consensus.cc:729] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab63f560bbb9499b9f8156a7a36b9923, State: Initialized, Role: FOLLOWER
I20260809 12:03:30.571578 3781 consensus_queue.cc:260] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.571775 3777 consensus_queue.cc:260] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.572192 3781 ts_tablet_manager.cc:1428] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260809 12:03:30.572191 3787 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.572103 3788 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.572458 3787 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:30.572479 3788 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:30.572918 3777 ts_tablet_manager.cc:1428] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 12:03:30.673257 3788 raft_consensus.cc:482] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:30.673525 3788 raft_consensus.cc:504] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.675143 3788 leader_election.cc:288] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763), bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841)
I20260809 12:03:30.682425 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.675505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36841 (local address 127.0.147.67:43431)
0809 12:03:30.675867 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.675890 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.676112 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.676580 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.676584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.676608 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:30.676866 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.676876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.678291 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.678299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.680436 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.680451 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.681839 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.681843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.681972 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.681995 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.682067 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.682157 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":44}
I20260809 12:03:30.682554 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.675584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:43431 (local address 127.0.147.66:36841)
0809 12:03:30.675803 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.675809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.676014 (+ 205us) server_negotiation.cc:407] Connection header received
0809 12:03:30.676326 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.676333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.676438 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.676555 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:30.677005 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.678169 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.680605 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.681679 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.681838 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.681914 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.682119 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.682251 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:03:30.683110 3475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "744fc6cf7a0f4fda9de3e335a1052b84" candidate_uuid: "28ca4b937a93434eb351ab6abd64edb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59be94d0d44819a079ce07df923fc0" is_pre_election: true
I20260809 12:03:30.683338 3772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.675595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:60493 (local address 127.0.147.65:46763)
0809 12:03:30.675809 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.675815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.676680 (+ 865us) server_negotiation.cc:407] Connection header received
0809 12:03:30.676868 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.676873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.676960 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.677046 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:30.677685 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.678743 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.681050 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.682239 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.682400 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.682468 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.682975 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.683101 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:03:30.683375 3475 raft_consensus.cc:2454] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28ca4b937a93434eb351ab6abd64edb3 in term 0.
I20260809 12:03:30.683454 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.675580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46763 (local address 127.0.147.67:60493)
0809 12:03:30.676536 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.676553 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.676714 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.677107 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.677113 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.677139 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:30.677473 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.677488 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.678908 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.678913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.680885 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.680895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.682479 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.682576 (+ 97us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.682820 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.682843 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.682945 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.683075 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":7,"thread_start_us":581,"threads_started":1}
I20260809 12:03:30.683934 3565 leader_election.cc:302] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ca4b937a93434eb351ab6abd64edb3, bc59be94d0d44819a079ce07df923fc0; no voters:
I20260809 12:03:30.684330 3788 raft_consensus.cc:2781] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:30.684481 3788 raft_consensus.cc:482] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:30.684602 3788 raft_consensus.cc:3037] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:30.685060 3262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "744fc6cf7a0f4fda9de3e335a1052b84" candidate_uuid: "28ca4b937a93434eb351ab6abd64edb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab63f560bbb9499b9f8156a7a36b9923" is_pre_election: true
I20260809 12:03:30.685465 3262 raft_consensus.cc:2454] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28ca4b937a93434eb351ab6abd64edb3 in term 0.
I20260809 12:03:30.687018 3788 raft_consensus.cc:504] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.687691 3788 leader_election.cc:288] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [CANDIDATE]: Term 1 election: Requested vote from peers ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763), bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841)
I20260809 12:03:30.688030 3262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "744fc6cf7a0f4fda9de3e335a1052b84" candidate_uuid: "28ca4b937a93434eb351ab6abd64edb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab63f560bbb9499b9f8156a7a36b9923"
I20260809 12:03:30.688133 3475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "744fc6cf7a0f4fda9de3e335a1052b84" candidate_uuid: "28ca4b937a93434eb351ab6abd64edb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59be94d0d44819a079ce07df923fc0"
I20260809 12:03:30.688223 3262 raft_consensus.cc:3037] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:30.688380 3475 raft_consensus.cc:3037] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:30.690538 3475 raft_consensus.cc:2454] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28ca4b937a93434eb351ab6abd64edb3 in term 1.
I20260809 12:03:30.690594 3262 raft_consensus.cc:2454] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28ca4b937a93434eb351ab6abd64edb3 in term 1.
I20260809 12:03:30.690912 3565 leader_election.cc:302] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ca4b937a93434eb351ab6abd64edb3, bc59be94d0d44819a079ce07df923fc0; no voters:
I20260809 12:03:30.691315 3788 raft_consensus.cc:2781] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:30.691879 3788 raft_consensus.cc:686] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 LEADER]: Becoming Leader. State: Replica: 28ca4b937a93434eb351ab6abd64edb3, State: Running, Role: LEADER
I20260809 12:03:30.692404 3788 consensus_queue.cc:237] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.695596 3807 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: TxnStatusTablet state changed. Reason: New leader 28ca4b937a93434eb351ab6abd64edb3. Latest consensus state: current_term: 1 leader_uuid: "28ca4b937a93434eb351ab6abd64edb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.696105 3807 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:03:30.696946 3812 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 12:03:30.697238 3812 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 12:03:30.697024 3037 catalog_manager.cc:5495] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67). New cstate: current_term: 1 leader_uuid: "28ca4b937a93434eb351ab6abd64edb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } health_report { overall_health: HEALTHY } } }
I20260809 12:03:30.724437 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.718037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45035 (local address 127.0.0.1:40748)
0809 12:03:30.718291 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.718311 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.718423 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.718912 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.718915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.718924 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:03:30.719073 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.719080 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.720617 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.720621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.721734 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.721747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.722723 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.722727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.722895 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.722963 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:03:30.723974 (+ 1011us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:03:30.723993 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.724064 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.724186 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 12:03:30.724494 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.718144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.0.147.67:45035)
0809 12:03:30.718333 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.718339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.718366 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:03:30.718615 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.718621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.718728 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.718885 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:30.719203 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.720455 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.721885 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.722568 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.722699 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.723179 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:30.723850 (+ 671us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:30.723995 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.724098 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.724183 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:03:30.732563 3475 raft_consensus.cc:1261] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Refusing update from remote peer 28ca4b937a93434eb351ab6abd64edb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:30.732537 3262 raft_consensus.cc:1261] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Refusing update from remote peer 28ca4b937a93434eb351ab6abd64edb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:30.733634 3807 consensus_queue.cc:1035] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:30.734128 3788 consensus_queue.cc:1035] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:30.747177 3787 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: TxnStatusTablet state changed. Reason: New leader 28ca4b937a93434eb351ab6abd64edb3. Latest consensus state: current_term: 1 leader_uuid: "28ca4b937a93434eb351ab6abd64edb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.747552 3787 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:30.749359 3788 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28ca4b937a93434eb351ab6abd64edb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.749660 3788 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:03:30.749605 3818 mvcc.cc:205] Tried to move back new op lower bound from 7316590635950080000 to 7316590635799625728. Current Snapshot: MvccSnapshot[applied={T|T < 7316590635950080000}]
I20260809 12:03:30.753108 3787 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28ca4b937a93434eb351ab6abd64edb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.753381 3787 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P ab63f560bbb9499b9f8156a7a36b9923: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:30.757946 3786 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: TxnStatusTablet state changed. Reason: New leader 28ca4b937a93434eb351ab6abd64edb3. Latest consensus state: current_term: 1 leader_uuid: "28ca4b937a93434eb351ab6abd64edb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.758282 3786 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:30.759743 3807 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28ca4b937a93434eb351ab6abd64edb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.760056 3807 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P 28ca4b937a93434eb351ab6abd64edb3: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:03:30.768872 3820 mvcc.cc:205] Tried to move back new op lower bound from 7316590635950080000 to 7316590635799625728. Current Snapshot: MvccSnapshot[applied={T|T < 7316590635950080000}]
I20260809 12:03:30.769938 3786 tablet_replica.cc:439] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28ca4b937a93434eb351ab6abd64edb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } } }
I20260809 12:03:30.770206 3786 tablet_replica.cc:442] T 744fc6cf7a0f4fda9de3e335a1052b84 P bc59be94d0d44819a079ce07df923fc0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:03:30.778600 3772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.770866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.0.147.65:46763)
0809 12:03:30.771034 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.771040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.771424 (+ 384us) server_negotiation.cc:407] Connection header received
0809 12:03:30.771743 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.771749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.771851 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.772006 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:30.772632 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.773519 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.775712 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.776456 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.776610 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.777165 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:30.777875 (+ 710us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:30.778019 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.778173 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.778266 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:03:30.797111 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.790530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:47739 (local address 127.0.147.67:45035)
0809 12:03:30.790775 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.790784 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.791262 (+ 478us) server_negotiation.cc:407] Connection header received
0809 12:03:30.791583 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.791589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.791685 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.791795 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:30.792376 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.793177 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.795070 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.795617 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.795757 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.796271 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:30.796381 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:30.796492 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.796656 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.796739 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:03:30.797195 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.790367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45035 (local address 127.0.147.65:47739)
0809 12:03:30.791134 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.791151 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.791359 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.791813 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.791818 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.791831 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:03:30.792202 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.792212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.793324 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.793333 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.794906 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.794920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.795834 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.795847 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.795976 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.796038 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:03:30.796494 (+ 456us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:03:30.796509 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.796628 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.796769 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":443,"threads_started":1}
I20260809 12:03:30.808911 3475 raft_consensus.cc:1261] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Refusing update from remote peer ab63f560bbb9499b9f8156a7a36b9923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:30.809103 3680 raft_consensus.cc:1261] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Refusing update from remote peer ab63f560bbb9499b9f8156a7a36b9923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:30.809792 3792 consensus_queue.cc:1035] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:30.810245 3787 consensus_queue.cc:1035] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:30.830936 3816 mvcc.cc:205] Tried to move back new op lower bound from 7316590636266373120 to 7316590634963853312. Current Snapshot: MvccSnapshot[applied={T|T < 7316590636266373120}]
I20260809 12:03:30.858248 3772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.850135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:53621 (local address 127.0.147.65:46763)
0809 12:03:30.850349 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.850358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.851479 (+ 1121us) server_negotiation.cc:407] Connection header received
0809 12:03:30.851719 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.851726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.851816 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.851930 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:30.852604 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.853937 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.855950 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.856697 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.856839 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.857339 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:30.857477 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:30.857540 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.857795 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.857875 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:03:30.859112 3842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.850146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46763 (local address 127.0.147.67:53621)
0809 12:03:30.851344 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.851360 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.851560 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.851951 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.851957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.851967 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:03:30.852372 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.852390 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.854086 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.854089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.855780 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.855794 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.856873 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.856877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.857041 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.857119 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:03:30.857625 (+ 506us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:03:30.857654 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.858486 (+ 832us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.858657 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":485,"threads_started":1}
I20260809 12:03:30.900280 3035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:30.900581 3035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:30.904903 3035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7: Sending DeleteTablet for 3 replicas of tablet e7c08a6ae7b946ee9f0ad8803f0d24df
I20260809 12:03:30.906163 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet e7c08a6ae7b946ee9f0ad8803f0d24df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:30 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:48064
I20260809 12:03:30.906478 3644 tablet_service.cc:1514] Processing DeleteTablet for tablet e7c08a6ae7b946ee9f0ad8803f0d24df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:30 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:40736
I20260809 12:03:30.906486 3434 tablet_service.cc:1514] Processing DeleteTablet for tablet e7c08a6ae7b946ee9f0ad8803f0d24df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:30 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:32820
I20260809 12:03:30.908849 3035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:03:30.909565 3035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:30.920235 3433 tablet_service.cc:1467] Processing CreateTablet for tablet ab02570d830b496a9585d9cf49cf70e1 (DEFAULT_TABLE table=test-table [id=99cd98614a464629bd7ad6d4aebed72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:30.920238 3224 tablet_service.cc:1467] Processing CreateTablet for tablet ab02570d830b496a9585d9cf49cf70e1 (DEFAULT_TABLE table=test-table [id=99cd98614a464629bd7ad6d4aebed72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:30.920957 3433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab02570d830b496a9585d9cf49cf70e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:30.920958 3224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab02570d830b496a9585d9cf49cf70e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:30.923653 3643 tablet_service.cc:1467] Processing CreateTablet for tablet ab02570d830b496a9585d9cf49cf70e1 (DEFAULT_TABLE table=test-table [id=99cd98614a464629bd7ad6d4aebed72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:30.924347 3643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab02570d830b496a9585d9cf49cf70e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:30.936086 3777 tablet_bootstrap.cc:492] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: Bootstrap starting.
I20260809 12:03:30.938149 3777 tablet_bootstrap.cc:654] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:30.938149 3776 tablet_bootstrap.cc:492] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: Bootstrap starting.
I20260809 12:03:30.940234 3776 tablet_bootstrap.cc:654] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:30.942356 3781 tablet_bootstrap.cc:492] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: Bootstrap starting.
I20260809 12:03:30.943730 3776 tablet_bootstrap.cc:492] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: No bootstrap required, opened a new log
I20260809 12:03:30.944010 3776 ts_tablet_manager.cc:1397] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 12:03:30.944671 3781 tablet_bootstrap.cc:654] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:30.945199 3776 raft_consensus.cc:348] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.945645 3776 raft_consensus.cc:374] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:30.945796 3776 raft_consensus.cc:729] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc59be94d0d44819a079ce07df923fc0, State: Initialized, Role: FOLLOWER
I20260809 12:03:30.946276 3776 consensus_queue.cc:260] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.948441 3776 ts_tablet_manager.cc:1428] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 12:03:30.958097 3777 tablet_bootstrap.cc:492] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: No bootstrap required, opened a new log
I20260809 12:03:30.958308 3777 ts_tablet_manager.cc:1397] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.000s
I20260809 12:03:30.959381 3781 tablet_bootstrap.cc:492] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: No bootstrap required, opened a new log
I20260809 12:03:30.959297 3777 raft_consensus.cc:348] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.959612 3781 ts_tablet_manager.cc:1397] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.005s
I20260809 12:03:30.959617 3777 raft_consensus.cc:374] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:30.959713 3777 raft_consensus.cc:729] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab63f560bbb9499b9f8156a7a36b9923, State: Initialized, Role: FOLLOWER
I20260809 12:03:30.960057 3777 consensus_queue.cc:260] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.960530 3781 raft_consensus.cc:348] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.960814 3781 raft_consensus.cc:374] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:30.960934 3781 raft_consensus.cc:729] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ca4b937a93434eb351ab6abd64edb3, State: Initialized, Role: FOLLOWER
I20260809 12:03:30.961274 3781 consensus_queue.cc:260] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.963933 3781 ts_tablet_manager.cc:1428] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 12:03:30.966846 3777 ts_tablet_manager.cc:1428] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260809 12:03:30.969791 3849 tablet_replica.cc:330] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: stopping tablet replica
I20260809 12:03:30.970314 3849 raft_consensus.cc:2229] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:30.970691 3849 raft_consensus.cc:2258] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:30.957010 3847 tablet_replica.cc:330] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: stopping tablet replica
I20260809 12:03:30.972675 3847 raft_consensus.cc:2229] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:30.973076 3847 raft_consensus.cc:2258] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:30.974018 3849 ts_tablet_manager.cc:1905] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:30.975117 3847 ts_tablet_manager.cc:1905] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:30.975184 3786 raft_consensus.cc:482] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:30.975354 3786 raft_consensus.cc:504] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:30.977159 3786 leader_election.cc:288] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763), 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035)
I20260809 12:03:30.977286 3846 tablet_replica.cc:330] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: stopping tablet replica
I20260809 12:03:30.977679 3846 raft_consensus.cc:2229] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:03:30.981653 3846 raft_consensus.cc:2258] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:30.983284 3849 ts_tablet_manager.cc:1918] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:03:30.983490 3849 log.cc:1196] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/e7c08a6ae7b946ee9f0ad8803f0d24df
I20260809 12:03:30.984153 3849 ts_tablet_manager.cc:1939] T e7c08a6ae7b946ee9f0ad8803f0d24df P 28ca4b937a93434eb351ab6abd64edb3: Deleting consensus metadata
I20260809 12:03:30.986202 3017 catalog_manager.cc:4841] TS 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035): tablet e7c08a6ae7b946ee9f0ad8803f0d24df (table test-table [id=dffd60cc371946739f5b021b74fe304b]) successfully deleted
I20260809 12:03:30.987373 3847 ts_tablet_manager.cc:1918] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:03:30.987586 3847 log.cc:1196] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/e7c08a6ae7b946ee9f0ad8803f0d24df
I20260809 12:03:30.988210 3847 ts_tablet_manager.cc:1939] T e7c08a6ae7b946ee9f0ad8803f0d24df P bc59be94d0d44819a079ce07df923fc0: Deleting consensus metadata
I20260809 12:03:30.992830 3846 ts_tablet_manager.cc:1905] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:30.995992 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.977961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:47799 (local address 127.0.147.67:45035)
0809 12:03:30.978230 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.978238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.986516 (+ 8278us) server_negotiation.cc:407] Connection header received
0809 12:03:30.986844 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:30.986853 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:30.986966 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:30.987045 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:30.988197 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.989689 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.992515 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.994321 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.994498 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.994608 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:30.995386 (+ 778us) server_negotiation.cc:299] Negotiation successful
0809 12:03:30.995602 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:03:30.996805 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.977611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45035 (local address 127.0.147.66:47799)
0809 12:03:30.986401 (+ 8790us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.986422 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.986640 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:30.987319 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:30.987327 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:30.987370 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:30.987942 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.987961 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.989878 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.989883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.992306 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:30.992323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:30.994871 (+ 2548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:30.994880 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:30.995143 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:30.995196 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 12:03:30.996250 (+ 1054us) client_negotiation.cc:240] Negotiation successful
0809 12:03:30.996441 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8650,"threads_started":1}
I20260809 12:03:30.997684 3679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab02570d830b496a9585d9cf49cf70e1" candidate_uuid: "bc59be94d0d44819a079ce07df923fc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ca4b937a93434eb351ab6abd64edb3" is_pre_election: true
I20260809 12:03:30.997977 3679 raft_consensus.cc:2454] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc59be94d0d44819a079ce07df923fc0 in term 0.
I20260809 12:03:30.998539 3360 leader_election.cc:302] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ca4b937a93434eb351ab6abd64edb3, bc59be94d0d44819a079ce07df923fc0; no voters:
I20260809 12:03:30.999006 3786 raft_consensus.cc:2781] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:30.999145 3786 raft_consensus.cc:482] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:30.999274 3786 raft_consensus.cc:3037] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:31.001389 3846 ts_tablet_manager.cc:1918] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:03:31.001575 3846 log.cc:1196] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/e7c08a6ae7b946ee9f0ad8803f0d24df
I20260809 12:03:31.001523 3786 raft_consensus.cc:504] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:31.002125 3846 ts_tablet_manager.cc:1939] T e7c08a6ae7b946ee9f0ad8803f0d24df P ab63f560bbb9499b9f8156a7a36b9923: Deleting consensus metadata
I20260809 12:03:31.002198 3786 leader_election.cc:288] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [CANDIDATE]: Term 1 election: Requested vote from peers ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763), 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035)
I20260809 12:03:31.002449 3017 catalog_manager.cc:4841] TS bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841): tablet e7c08a6ae7b946ee9f0ad8803f0d24df (table test-table [id=dffd60cc371946739f5b021b74fe304b]) successfully deleted
I20260809 12:03:31.003209 3679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab02570d830b496a9585d9cf49cf70e1" candidate_uuid: "bc59be94d0d44819a079ce07df923fc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ca4b937a93434eb351ab6abd64edb3"
I20260809 12:03:31.003466 3679 raft_consensus.cc:3037] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:31.006749 3679 raft_consensus.cc:2454] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc59be94d0d44819a079ce07df923fc0 in term 1.
I20260809 12:03:31.008874 3360 leader_election.cc:302] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ca4b937a93434eb351ab6abd64edb3, bc59be94d0d44819a079ce07df923fc0; no voters:
I20260809 12:03:31.009179 3786 raft_consensus.cc:2781] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:31.010897 3772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.979609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:53645 (local address 127.0.147.65:46763)
0809 12:03:30.979811 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:30.979818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:30.996854 (+ 17036us) server_negotiation.cc:407] Connection header received
0809 12:03:31.002935 (+ 6081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:31.002940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:31.003032 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:31.003104 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:31.003873 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.005016 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.007573 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.008980 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.009145 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:31.009221 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:31.010440 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0809 12:03:31.010599 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 12:03:31.011755 3854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:30.979132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46763 (local address 127.0.147.66:53645)
0809 12:03:30.996737 (+ 17605us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:30.996757 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:30.997368 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:31.003258 (+ 5890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:31.003261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:31.003283 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:31.003708 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:31.003719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.005196 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.005202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:31.007341 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:31.007359 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.010033 (+ 2674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.010040 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:31.010241 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:31.010276 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:03:31.011219 (+ 943us) client_negotiation.cc:240] Negotiation successful
0809 12:03:31.011369 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17462,"thread_start_us":7408,"threads_started":1}
I20260809 12:03:31.012554 3262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab02570d830b496a9585d9cf49cf70e1" candidate_uuid: "bc59be94d0d44819a079ce07df923fc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab63f560bbb9499b9f8156a7a36b9923" is_pre_election: true
I20260809 12:03:31.012739 3261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab02570d830b496a9585d9cf49cf70e1" candidate_uuid: "bc59be94d0d44819a079ce07df923fc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab63f560bbb9499b9f8156a7a36b9923"
I20260809 12:03:31.012820 3262 raft_consensus.cc:2454] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc59be94d0d44819a079ce07df923fc0 in term 0.
I20260809 12:03:31.016528 3018 catalog_manager.cc:4841] TS ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763): tablet e7c08a6ae7b946ee9f0ad8803f0d24df (table test-table [id=dffd60cc371946739f5b021b74fe304b]) successfully deleted
I20260809 12:03:31.016865 3786 raft_consensus.cc:686] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 1 LEADER]: Becoming Leader. State: Replica: bc59be94d0d44819a079ce07df923fc0, State: Running, Role: LEADER
I20260809 12:03:31.017582 3786 consensus_queue.cc:237] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:31.021407 3035 catalog_manager.cc:5495] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc59be94d0d44819a079ce07df923fc0 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "bc59be94d0d44819a079ce07df923fc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } health_report { overall_health: UNKNOWN } } }
I20260809 12:03:31.066395 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:31.059234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.0.147.66:36841)
0809 12:03:31.059413 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:31.059419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:31.059438 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:03:31.059620 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:31.059625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:31.059732 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:31.059894 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:31.060356 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.061518 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.063655 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.064337 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.064472 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:31.065081 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:31.065718 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:31.065861 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:31.066018 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 12:03:31.066114 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:03:31.091116 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:31.081335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:33639 (local address 127.0.147.67:45035)
0809 12:03:31.081515 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:31.081521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:31.084327 (+ 2806us) server_negotiation.cc:407] Connection header received
0809 12:03:31.084546 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:31.084553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:31.084657 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:31.084773 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:31.085393 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.086660 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.088717 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.089441 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.089592 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:31.090206 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:31.090390 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:31.090503 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:31.090641 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 12:03:31.090736 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:03:31.091169 3861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:31.081227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45035 (local address 127.0.147.66:33639)
0809 12:03:31.084147 (+ 2920us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:31.084168 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:31.084376 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:31.084800 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:31.084805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:31.084818 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:03:31.085194 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:31.085209 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.086830 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.086836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:31.088559 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:31.088575 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.089654 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.089661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:31.089858 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:31.089927 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:03:31.090502 (+ 575us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:03:31.090515 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 12:03:31.090603 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 12:03:31.090750 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2740,"thread_start_us":2543,"threads_started":1}
I20260809 12:03:31.103314 3679 raft_consensus.cc:1261] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Refusing update from remote peer bc59be94d0d44819a079ce07df923fc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:31.103435 3261 raft_consensus.cc:3037] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:31.107614 3261 raft_consensus.cc:1261] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Refusing update from remote peer bc59be94d0d44819a079ce07df923fc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:31.107748 3856 consensus_queue.cc:1035] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:31.108351 3786 consensus_queue.cc:1035] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:31.160691 3842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:31.152082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36841 (local address 127.0.147.67:42821)
0809 12:03:31.152266 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:31.152289 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:31.152415 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:31.153917 (+ 1502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:31.153929 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:31.153944 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:03:31.154174 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:31.154188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.155880 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.155887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:31.157809 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:31.157827 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.159114 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.159122 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:31.159353 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:31.159416 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:03:31.160014 (+ 598us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:03:31.160025 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 12:03:31.160110 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 12:03:31.160271 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 12:03:31.161614 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:31.153161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:42821 (local address 127.0.147.66:36841)
0809 12:03:31.153363 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:31.153370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:31.153391 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:03:31.153470 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:31.153477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:31.153567 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:31.153671 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:31.154411 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.155692 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.158027 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:31.158818 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:31.158961 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:31.159676 (+ 715us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:31.159835 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:31.159926 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:31.161198 (+ 1272us) server_negotiation.cc:299] Negotiation successful
0809 12:03:31.161290 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 12:03:31.162822 3820 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 12:03:32.792121 3035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:32.792397 3035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:32.797991 3035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7: Sending DeleteTablet for 3 replicas of tablet ab02570d830b496a9585d9cf49cf70e1
I20260809 12:03:32.799068 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet ab02570d830b496a9585d9cf49cf70e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:32 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:48064
I20260809 12:03:32.799147 3434 tablet_service.cc:1514] Processing DeleteTablet for tablet ab02570d830b496a9585d9cf49cf70e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:32 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:32820
I20260809 12:03:32.799371 3644 tablet_service.cc:1514] Processing DeleteTablet for tablet ab02570d830b496a9585d9cf49cf70e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:32 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:40736
I20260809 12:03:32.814574 3908 tablet_replica.cc:330] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: stopping tablet replica
I20260809 12:03:32.815133 3908 raft_consensus.cc:2229] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:32.815300 3906 tablet_replica.cc:330] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: stopping tablet replica
I20260809 12:03:32.815677 3908 raft_consensus.cc:2258] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:32.815752 3906 raft_consensus.cc:2229] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:32.816135 3906 raft_consensus.cc:2258] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:32.817189 3908 ts_tablet_manager.cc:1905] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:32.819161 3906 ts_tablet_manager.cc:1905] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:32.820513 3904 tablet_replica.cc:330] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: stopping tablet replica
I20260809 12:03:32.820863 3904 raft_consensus.cc:2229] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:03:32.821777 3904 raft_consensus.cc:2258] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:32.824321 3904 ts_tablet_manager.cc:1905] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:32.827387 3908 ts_tablet_manager.cc:1918] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:03:32.827797 3908 log.cc:1196] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/ab02570d830b496a9585d9cf49cf70e1
I20260809 12:03:32.828678 3908 ts_tablet_manager.cc:1939] T ab02570d830b496a9585d9cf49cf70e1 P ab63f560bbb9499b9f8156a7a36b9923: Deleting consensus metadata
I20260809 12:03:32.830060 3018 catalog_manager.cc:4841] TS ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763): tablet ab02570d830b496a9585d9cf49cf70e1 (table test-table [id=99cd98614a464629bd7ad6d4aebed72f]) successfully deleted
I20260809 12:03:32.830266 3906 ts_tablet_manager.cc:1918] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:03:32.830577 3906 log.cc:1196] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/ab02570d830b496a9585d9cf49cf70e1
I20260809 12:03:32.831096 3906 ts_tablet_manager.cc:1939] T ab02570d830b496a9585d9cf49cf70e1 P 28ca4b937a93434eb351ab6abd64edb3: Deleting consensus metadata
I20260809 12:03:32.832424 3017 catalog_manager.cc:4841] TS 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035): tablet ab02570d830b496a9585d9cf49cf70e1 (table test-table [id=99cd98614a464629bd7ad6d4aebed72f]) successfully deleted
I20260809 12:03:32.832959 3904 ts_tablet_manager.cc:1918] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:03:32.833206 3904 log.cc:1196] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/ab02570d830b496a9585d9cf49cf70e1
I20260809 12:03:32.833600 3904 ts_tablet_manager.cc:1939] T ab02570d830b496a9585d9cf49cf70e1 P bc59be94d0d44819a079ce07df923fc0: Deleting consensus metadata
I20260809 12:03:32.834579 3017 catalog_manager.cc:4841] TS bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841): tablet ab02570d830b496a9585d9cf49cf70e1 (table test-table [id=99cd98614a464629bd7ad6d4aebed72f]) successfully deleted
Aug 09 12:03:32 dist-test-slave-sb2z krb5kdc[2977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:03:32 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:32.878055 3918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:32.857788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.0.147.126:35755)
0809 12:03:32.858549 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:32.858581 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:32.858617 (+ 36us) server_negotiation.cc:407] Connection header received
0809 12:03:32.858861 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:32.858867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:32.858946 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:32.859098 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:32.860715 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:32.861557 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:32.862886 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:32.863329 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:32.863474 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:32.871275 (+ 7801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:32.871281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:32.871285 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:32.871321 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:32.875127 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:32.875786 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:32.875790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:32.875793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:32.875981 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:32.876306 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:32.876309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:32.876311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:32.876765 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:32.876996 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:32.877259 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 12:03:32.877462 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":383,"threads_started":1}
I20260809 12:03:32.882223 3035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:03:32.882901 3035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:32.889973 3225 tablet_service.cc:1467] Processing CreateTablet for tablet 7d33d064ac514283b5f65bc218a0e13f (DEFAULT_TABLE table=test-table [id=aa1a55e243df4ab294e6951dda54aad1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:32.890398 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d33d064ac514283b5f65bc218a0e13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:32.890316 3434 tablet_service.cc:1467] Processing CreateTablet for tablet 7d33d064ac514283b5f65bc218a0e13f (DEFAULT_TABLE table=test-table [id=aa1a55e243df4ab294e6951dda54aad1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:32.890830 3434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d33d064ac514283b5f65bc218a0e13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:32.892118 3644 tablet_service.cc:1467] Processing CreateTablet for tablet 7d33d064ac514283b5f65bc218a0e13f (DEFAULT_TABLE table=test-table [id=aa1a55e243df4ab294e6951dda54aad1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:32.892642 3644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d33d064ac514283b5f65bc218a0e13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:32.897329 3919 tablet_bootstrap.cc:492] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: Bootstrap starting.
I20260809 12:03:32.899646 3919 tablet_bootstrap.cc:654] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:32.901373 3922 tablet_bootstrap.cc:492] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: Bootstrap starting.
I20260809 12:03:32.903227 3920 tablet_bootstrap.cc:492] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: Bootstrap starting.
I20260809 12:03:32.903322 3919 tablet_bootstrap.cc:492] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: No bootstrap required, opened a new log
I20260809 12:03:32.903504 3919 ts_tablet_manager.cc:1397] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260809 12:03:32.904529 3922 tablet_bootstrap.cc:654] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:32.904539 3919 raft_consensus.cc:348] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:32.904978 3919 raft_consensus.cc:374] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:32.905165 3919 raft_consensus.cc:729] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc59be94d0d44819a079ce07df923fc0, State: Initialized, Role: FOLLOWER
I20260809 12:03:32.905289 3920 tablet_bootstrap.cc:654] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:32.905525 3919 consensus_queue.cc:260] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:32.906270 3919 ts_tablet_manager.cc:1428] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 12:03:32.908471 3922 tablet_bootstrap.cc:492] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: No bootstrap required, opened a new log
I20260809 12:03:32.908653 3922 ts_tablet_manager.cc:1397] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 12:03:32.908860 3920 tablet_bootstrap.cc:492] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: No bootstrap required, opened a new log
I20260809 12:03:32.909071 3920 ts_tablet_manager.cc:1397] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 12:03:32.909667 3922 raft_consensus.cc:348] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:32.909973 3922 raft_consensus.cc:374] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:32.909996 3920 raft_consensus.cc:348] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:32.910138 3922 raft_consensus.cc:729] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ca4b937a93434eb351ab6abd64edb3, State: Initialized, Role: FOLLOWER
I20260809 12:03:32.910279 3920 raft_consensus.cc:374] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:32.910418 3920 raft_consensus.cc:729] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab63f560bbb9499b9f8156a7a36b9923, State: Initialized, Role: FOLLOWER
I20260809 12:03:32.910422 3922 consensus_queue.cc:260] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:32.910710 3920 consensus_queue.cc:260] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:32.911125 3922 ts_tablet_manager.cc:1428] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 12:03:32.912521 3920 ts_tablet_manager.cc:1428] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 12:03:32.953862 3891 raft_consensus.cc:482] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:32.954118 3891 raft_consensus.cc:504] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:32.955049 3891 leader_election.cc:288] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763), 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035)
I20260809 12:03:32.955709 3680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d33d064ac514283b5f65bc218a0e13f" candidate_uuid: "bc59be94d0d44819a079ce07df923fc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ca4b937a93434eb351ab6abd64edb3" is_pre_election: true
I20260809 12:03:32.955753 3259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d33d064ac514283b5f65bc218a0e13f" candidate_uuid: "bc59be94d0d44819a079ce07df923fc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab63f560bbb9499b9f8156a7a36b9923" is_pre_election: true
I20260809 12:03:32.956036 3259 raft_consensus.cc:2454] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc59be94d0d44819a079ce07df923fc0 in term 0.
I20260809 12:03:32.956120 3680 raft_consensus.cc:2454] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc59be94d0d44819a079ce07df923fc0 in term 0.
I20260809 12:03:32.956585 3362 leader_election.cc:302] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab63f560bbb9499b9f8156a7a36b9923, bc59be94d0d44819a079ce07df923fc0; no voters:
I20260809 12:03:32.956977 3891 raft_consensus.cc:2781] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:32.957149 3891 raft_consensus.cc:482] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:32.957293 3891 raft_consensus.cc:3037] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:32.960008 3891 raft_consensus.cc:504] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:32.960598 3891 leader_election.cc:288] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [CANDIDATE]: Term 1 election: Requested vote from peers ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763), 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035)
I20260809 12:03:32.961148 3259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d33d064ac514283b5f65bc218a0e13f" candidate_uuid: "bc59be94d0d44819a079ce07df923fc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab63f560bbb9499b9f8156a7a36b9923"
I20260809 12:03:32.961258 3680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d33d064ac514283b5f65bc218a0e13f" candidate_uuid: "bc59be94d0d44819a079ce07df923fc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ca4b937a93434eb351ab6abd64edb3"
I20260809 12:03:32.961395 3259 raft_consensus.cc:3037] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:32.961503 3680 raft_consensus.cc:3037] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:32.964870 3259 raft_consensus.cc:2454] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc59be94d0d44819a079ce07df923fc0 in term 1.
I20260809 12:03:32.964879 3680 raft_consensus.cc:2454] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc59be94d0d44819a079ce07df923fc0 in term 1.
I20260809 12:03:32.965451 3360 leader_election.cc:302] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ca4b937a93434eb351ab6abd64edb3, bc59be94d0d44819a079ce07df923fc0; no voters:
I20260809 12:03:32.965837 3891 raft_consensus.cc:2781] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:32.966020 3891 raft_consensus.cc:686] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 1 LEADER]: Becoming Leader. State: Replica: bc59be94d0d44819a079ce07df923fc0, State: Running, Role: LEADER
I20260809 12:03:32.966338 3891 consensus_queue.cc:237] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:32.969250 3037 catalog_manager.cc:5495] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc59be94d0d44819a079ce07df923fc0 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "bc59be94d0d44819a079ce07df923fc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } health_report { overall_health: UNKNOWN } } }
I20260809 12:03:32.994752 3937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:32.986703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.0.147.66:36841)
0809 12:03:32.987609 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:32.987613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:32.987633 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:03:32.987682 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:32.987685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:32.987740 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:32.987833 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:32.988461 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:32.989535 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:32.992260 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:32.992981 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:32.993086 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:32.993860 (+ 774us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:32.993988 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:32.994057 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:32.994263 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 12:03:32.994323 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":619,"threads_started":1}
I20260809 12:03:33.004033 3680 raft_consensus.cc:1261] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Refusing update from remote peer bc59be94d0d44819a079ce07df923fc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:33.004326 3259 raft_consensus.cc:1261] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Refusing update from remote peer bc59be94d0d44819a079ce07df923fc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:33.004652 3891 consensus_queue.cc:1035] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:33.005038 3890 consensus_queue.cc:1035] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:33.059932 3035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:33.060209 3035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:33.063318 3035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7: Sending DeleteTablet for 3 replicas of tablet 7d33d064ac514283b5f65bc218a0e13f
I20260809 12:03:33.064344 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d33d064ac514283b5f65bc218a0e13f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:33 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:48064
I20260809 12:03:33.064561 3434 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d33d064ac514283b5f65bc218a0e13f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:33 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:32820
I20260809 12:03:33.064711 3908 tablet_replica.cc:330] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: stopping tablet replica
I20260809 12:03:33.064916 3904 tablet_replica.cc:330] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: stopping tablet replica
I20260809 12:03:33.064987 3908 raft_consensus.cc:2229] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:33.064929 3644 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d33d064ac514283b5f65bc218a0e13f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:33 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:40736
I20260809 12:03:33.065207 3904 raft_consensus.cc:2229] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:03:33.065232 3908 raft_consensus.cc:2258] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:33.065405 3906 tablet_replica.cc:330] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: stopping tablet replica
I20260809 12:03:33.065681 3904 raft_consensus.cc:2258] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:33.065701 3906 raft_consensus.cc:2229] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:33.065994 3906 raft_consensus.cc:2258] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:33.066670 3908 ts_tablet_manager.cc:1905] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:33.066952 3904 ts_tablet_manager.cc:1905] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:33.067162 3906 ts_tablet_manager.cc:1905] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:33.067705 3035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:03:33.068480 3035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:33.075182 3908 ts_tablet_manager.cc:1918] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:03:33.075414 3908 log.cc:1196] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/7d33d064ac514283b5f65bc218a0e13f
I20260809 12:03:33.075810 3906 ts_tablet_manager.cc:1918] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:03:33.076078 3906 log.cc:1196] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/7d33d064ac514283b5f65bc218a0e13f
I20260809 12:03:33.076493 3225 tablet_service.cc:1467] Processing CreateTablet for tablet 4fac5d8f7f8244b6a4fab5a2e2da0830 (DEFAULT_TABLE table=test-table [id=0a5e9d4d129c4412bea473898b521879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:33.077026 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fac5d8f7f8244b6a4fab5a2e2da0830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:33.077310 3904 ts_tablet_manager.cc:1918] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:03:33.077894 3434 tablet_service.cc:1467] Processing CreateTablet for tablet 4fac5d8f7f8244b6a4fab5a2e2da0830 (DEFAULT_TABLE table=test-table [id=0a5e9d4d129c4412bea473898b521879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:33.078406 3434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fac5d8f7f8244b6a4fab5a2e2da0830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:33.078372 3644 tablet_service.cc:1467] Processing CreateTablet for tablet 4fac5d8f7f8244b6a4fab5a2e2da0830 (DEFAULT_TABLE table=test-table [id=0a5e9d4d129c4412bea473898b521879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:33.079035 3644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fac5d8f7f8244b6a4fab5a2e2da0830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:33.079756 3906 ts_tablet_manager.cc:1939] T 7d33d064ac514283b5f65bc218a0e13f P 28ca4b937a93434eb351ab6abd64edb3: Deleting consensus metadata
I20260809 12:03:33.082329 3017 catalog_manager.cc:4841] TS 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035): tablet 7d33d064ac514283b5f65bc218a0e13f (table test-table [id=aa1a55e243df4ab294e6951dda54aad1]) successfully deleted
I20260809 12:03:33.085325 3920 tablet_bootstrap.cc:492] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: Bootstrap starting.
I20260809 12:03:33.085657 3908 ts_tablet_manager.cc:1939] T 7d33d064ac514283b5f65bc218a0e13f P ab63f560bbb9499b9f8156a7a36b9923: Deleting consensus metadata
I20260809 12:03:33.085839 3919 tablet_bootstrap.cc:492] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: Bootstrap starting.
I20260809 12:03:33.086793 3018 catalog_manager.cc:4841] TS ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763): tablet 7d33d064ac514283b5f65bc218a0e13f (table test-table [id=aa1a55e243df4ab294e6951dda54aad1]) successfully deleted
I20260809 12:03:33.088052 3922 tablet_bootstrap.cc:492] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: Bootstrap starting.
I20260809 12:03:33.088256 3920 tablet_bootstrap.cc:654] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:33.088577 3919 tablet_bootstrap.cc:654] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:33.089874 3904 log.cc:1196] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/7d33d064ac514283b5f65bc218a0e13f
I20260809 12:03:33.090767 3904 ts_tablet_manager.cc:1939] T 7d33d064ac514283b5f65bc218a0e13f P bc59be94d0d44819a079ce07df923fc0: Deleting consensus metadata
I20260809 12:03:33.091571 3922 tablet_bootstrap.cc:654] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:33.093755 3017 catalog_manager.cc:4841] TS bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841): tablet 7d33d064ac514283b5f65bc218a0e13f (table test-table [id=aa1a55e243df4ab294e6951dda54aad1]) successfully deleted
I20260809 12:03:33.095357 3920 tablet_bootstrap.cc:492] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: No bootstrap required, opened a new log
I20260809 12:03:33.095664 3920 ts_tablet_manager.cc:1397] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.001s
I20260809 12:03:33.096503 3920 raft_consensus.cc:348] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:33.096776 3922 tablet_bootstrap.cc:492] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: No bootstrap required, opened a new log
I20260809 12:03:33.096764 3919 tablet_bootstrap.cc:492] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: No bootstrap required, opened a new log
I20260809 12:03:33.096848 3920 raft_consensus.cc:374] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:33.097033 3922 ts_tablet_manager.cc:1397] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260809 12:03:33.097057 3919 ts_tablet_manager.cc:1397] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260809 12:03:33.097014 3920 raft_consensus.cc:729] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab63f560bbb9499b9f8156a7a36b9923, State: Initialized, Role: FOLLOWER
I20260809 12:03:33.097503 3920 consensus_queue.cc:260] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:33.098141 3922 raft_consensus.cc:348] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:33.098182 3919 raft_consensus.cc:348] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:33.098549 3919 raft_consensus.cc:374] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:33.098773 3919 raft_consensus.cc:729] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc59be94d0d44819a079ce07df923fc0, State: Initialized, Role: FOLLOWER
I20260809 12:03:33.098469 3922 raft_consensus.cc:374] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:33.098955 3922 raft_consensus.cc:729] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ca4b937a93434eb351ab6abd64edb3, State: Initialized, Role: FOLLOWER
I20260809 12:03:33.098865 3920 ts_tablet_manager.cc:1428] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 12:03:33.099140 3919 consensus_queue.cc:260] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:33.099947 3919 ts_tablet_manager.cc:1428] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 12:03:33.099303 3922 consensus_queue.cc:260] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:33.100610 3922 ts_tablet_manager.cc:1428] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 12:03:33.453593 3894 raft_consensus.cc:482] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:33.453859 3894 raft_consensus.cc:504] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:33.454811 3894 leader_election.cc:288] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763), bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841)
I20260809 12:03:33.455330 3259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fac5d8f7f8244b6a4fab5a2e2da0830" candidate_uuid: "28ca4b937a93434eb351ab6abd64edb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab63f560bbb9499b9f8156a7a36b9923" is_pre_election: true
I20260809 12:03:33.455698 3259 raft_consensus.cc:2454] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28ca4b937a93434eb351ab6abd64edb3 in term 0.
I20260809 12:03:33.455643 3475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fac5d8f7f8244b6a4fab5a2e2da0830" candidate_uuid: "28ca4b937a93434eb351ab6abd64edb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59be94d0d44819a079ce07df923fc0" is_pre_election: true
I20260809 12:03:33.455929 3475 raft_consensus.cc:2454] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28ca4b937a93434eb351ab6abd64edb3 in term 0.
I20260809 12:03:33.456321 3568 leader_election.cc:302] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ca4b937a93434eb351ab6abd64edb3, ab63f560bbb9499b9f8156a7a36b9923; no voters:
I20260809 12:03:33.456766 3894 raft_consensus.cc:2781] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:33.456976 3894 raft_consensus.cc:482] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:33.457124 3894 raft_consensus.cc:3037] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:33.459844 3894 raft_consensus.cc:504] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:33.460592 3894 leader_election.cc:288] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [CANDIDATE]: Term 1 election: Requested vote from peers ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763), bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841)
I20260809 12:03:33.461102 3475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fac5d8f7f8244b6a4fab5a2e2da0830" candidate_uuid: "28ca4b937a93434eb351ab6abd64edb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59be94d0d44819a079ce07df923fc0"
I20260809 12:03:33.461133 3259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fac5d8f7f8244b6a4fab5a2e2da0830" candidate_uuid: "28ca4b937a93434eb351ab6abd64edb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab63f560bbb9499b9f8156a7a36b9923"
I20260809 12:03:33.461337 3475 raft_consensus.cc:3037] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:33.461376 3259 raft_consensus.cc:3037] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:33.463973 3475 raft_consensus.cc:2454] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28ca4b937a93434eb351ab6abd64edb3 in term 1.
I20260809 12:03:33.464289 3259 raft_consensus.cc:2454] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28ca4b937a93434eb351ab6abd64edb3 in term 1.
I20260809 12:03:33.464416 3565 leader_election.cc:302] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ca4b937a93434eb351ab6abd64edb3, bc59be94d0d44819a079ce07df923fc0; no voters:
I20260809 12:03:33.464840 3894 raft_consensus.cc:2781] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:33.465035 3894 raft_consensus.cc:686] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 LEADER]: Becoming Leader. State: Replica: 28ca4b937a93434eb351ab6abd64edb3, State: Running, Role: LEADER
I20260809 12:03:33.465416 3894 consensus_queue.cc:237] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } }
I20260809 12:03:33.468744 3037 catalog_manager.cc:5495] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67). New cstate: current_term: 1 leader_uuid: "28ca4b937a93434eb351ab6abd64edb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28ca4b937a93434eb351ab6abd64edb3" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45035 } health_report { overall_health: HEALTHY } } }
I20260809 12:03:33.554183 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:33.547649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.0.147.67:45035)
0809 12:03:33.548451 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:33.548457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:33.548480 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:03:33.548536 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:33.548541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:33.548615 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:33.548721 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:33.549381 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:33.550354 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:33.552044 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:33.552642 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:33.552784 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:33.553339 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:33.553441 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:33.553543 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:33.553748 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 12:03:33.553826 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":543,"threads_started":1}
I20260809 12:03:33.569754 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:33.563658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:55987 (local address 127.0.147.67:45035)
0809 12:03:33.563955 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:33.563962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:33.564473 (+ 511us) server_negotiation.cc:407] Connection header received
0809 12:03:33.564678 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:33.564682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:33.564747 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:33.564842 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:33.565390 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:33.566189 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:33.567800 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:33.568343 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:33.568482 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:33.568928 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:33.569043 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:33.569143 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:33.569292 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 12:03:33.569381 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":60}
I20260809 12:03:33.570120 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:33.563360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45035 (local address 127.0.147.67:55987)
0809 12:03:33.564365 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:33.564380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:33.564558 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:33.564853 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:33.564857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:33.564867 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:03:33.565252 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:33.565261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:33.566359 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:33.566366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:33.567659 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:33.567671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:33.568561 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:33.568568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:33.568739 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:33.568772 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:03:33.569144 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:03:33.569157 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 12:03:33.569275 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 12:03:33.569422 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":1,"thread_start_us":670,"threads_started":1}
I20260809 12:03:33.578934 3475 raft_consensus.cc:1261] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Refusing update from remote peer 28ca4b937a93434eb351ab6abd64edb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:33.578974 3259 raft_consensus.cc:1261] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Refusing update from remote peer 28ca4b937a93434eb351ab6abd64edb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:33.579583 3873 consensus_queue.cc:1035] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab63f560bbb9499b9f8156a7a36b9923" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:33.580015 3952 consensus_queue.cc:1035] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc59be94d0d44819a079ce07df923fc0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:03:33.601387 3870 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 12:03:35.214798 3037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:35.215106 3037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:35.220489 3037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 796728bb827b425e9b6038ca93cb0de7: Sending DeleteTablet for 3 replicas of tablet 4fac5d8f7f8244b6a4fab5a2e2da0830
I20260809 12:03:35.221557 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fac5d8f7f8244b6a4fab5a2e2da0830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:35 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:48064
I20260809 12:03:35.221859 3434 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fac5d8f7f8244b6a4fab5a2e2da0830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:35 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:32820
I20260809 12:03:35.222359 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 3128
I20260809 12:03:35.223174 3997 tablet_replica.cc:330] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: stopping tablet replica
I20260809 12:03:35.223466 3997 raft_consensus.cc:2229] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:35.223927 3997 raft_consensus.cc:2258] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:35.225868 3998 tablet_replica.cc:330] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: stopping tablet replica
I20260809 12:03:35.223449 3643 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fac5d8f7f8244b6a4fab5a2e2da0830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:35 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:40736
I20260809 12:03:35.226167 3998 raft_consensus.cc:2229] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:35.226548 3998 raft_consensus.cc:2258] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:35.227499 3997 ts_tablet_manager.cc:1905] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:35.232318 3998 ts_tablet_manager.cc:1905] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:35.236140 4002 tablet_replica.cc:330] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: stopping tablet replica
I20260809 12:03:35.236414 4002 raft_consensus.cc:2229] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:03:35.237118 3997 ts_tablet_manager.cc:1918] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:03:35.237129 4002 raft_consensus.cc:2258] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:35.237807 3997 log.cc:1196] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/4fac5d8f7f8244b6a4fab5a2e2da0830
I20260809 12:03:35.238533 4002 ts_tablet_manager.cc:1905] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:35.238852 3997 ts_tablet_manager.cc:1939] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P ab63f560bbb9499b9f8156a7a36b9923: Deleting consensus metadata
I20260809 12:03:35.240025 3018 catalog_manager.cc:4841] TS ab63f560bbb9499b9f8156a7a36b9923 (127.0.147.65:46763): tablet 4fac5d8f7f8244b6a4fab5a2e2da0830 (table test-table [id=0a5e9d4d129c4412bea473898b521879]) successfully deleted
Aug 09 12:03:35 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:35.247190 3998 ts_tablet_manager.cc:1918] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:03:35.247483 3998 log.cc:1196] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/4fac5d8f7f8244b6a4fab5a2e2da0830
I20260809 12:03:35.247917 4002 ts_tablet_manager.cc:1918] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:03:35.248167 3998 ts_tablet_manager.cc:1939] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P bc59be94d0d44819a079ce07df923fc0: Deleting consensus metadata
I20260809 12:03:35.248167 4002 log.cc:1196] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/4fac5d8f7f8244b6a4fab5a2e2da0830
I20260809 12:03:35.249024 4002 ts_tablet_manager.cc:1939] T 4fac5d8f7f8244b6a4fab5a2e2da0830 P 28ca4b937a93434eb351ab6abd64edb3: Deleting consensus metadata
I20260809 12:03:35.249452 3017 catalog_manager.cc:4841] TS bc59be94d0d44819a079ce07df923fc0 (127.0.147.66:36841): tablet 4fac5d8f7f8244b6a4fab5a2e2da0830 (table test-table [id=0a5e9d4d129c4412bea473898b521879]) successfully deleted
I20260809 12:03:35.250542 3017 catalog_manager.cc:4841] TS 28ca4b937a93434eb351ab6abd64edb3 (127.0.147.67:45035): tablet 4fac5d8f7f8244b6a4fab5a2e2da0830 (table test-table [id=0a5e9d4d129c4412bea473898b521879]) successfully deleted
I20260809 12:03:35.256887 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:35.224577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.0.147.65:46763)
0809 12:03:35.226184 (+ 1607us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:35.226192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:35.226245 (+ 53us) server_negotiation.cc:407] Connection header received
0809 12:03:35.226306 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:35.226310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:35.226396 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:35.226614 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:35.230094 (+ 3480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:35.232492 (+ 2398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:35.235828 (+ 3336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:35.236526 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:35.236655 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:35.248477 (+ 11822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:35.248512 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:35.248543 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:35.248612 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:35.253361 (+ 4749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:35.254479 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:35.254487 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:35.254489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:35.254653 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:35.255107 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:35.255110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:35.255112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:35.255517 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:35.255910 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:35.256307 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 12:03:35.256571 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1426,"thread_start_us":1692,"threads_started":1}
W20260809 12:03:35.257723 3330 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58324
W20260809 12:03:35.258442 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 3128: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:03:35.258706 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 3128
I20260809 12:03:35.293771 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 3342
Aug 09 12:03:35 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:03:35.313723 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:35.294591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.0.147.66:36841)
0809 12:03:35.295336 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:35.295340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:35.295357 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:03:35.295411 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:35.295415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:35.295472 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:35.295678 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:35.297975 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:35.298725 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:35.300823 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:35.301288 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:35.301405 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:35.306419 (+ 5014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:35.306432 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:35.306447 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:35.306511 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:35.310815 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:35.311573 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:35.311584 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:35.311589 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:35.311727 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:35.312108 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:35.312114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:35.312117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:35.312466 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:35.312791 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:35.313114 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 12:03:35.313345 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":517,"threads_started":1}
W20260809 12:03:35.314419 3531 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53058
W20260809 12:03:35.315061 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 3342: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:03:35.315235 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 3342
I20260809 12:03:35.351118 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 3541
Aug 09 12:03:35 dist-test-slave-sb2z krb5kdc[2977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:03:35.367331 4005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:35.351930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.0.147.67:45035)
0809 12:03:35.352536 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:35.352540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:35.352557 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:03:35.352612 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:35.352616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:35.352673 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:35.352768 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:35.354148 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:35.354904 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:35.356247 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:35.356647 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:35.356740 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:35.361258 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:35.361283 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:35.361322 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:35.361373 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:35.364519 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:35.365181 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:35.365197 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:35.365199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:35.365344 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:35.365750 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:35.365756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:35.365759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:35.366109 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:35.366416 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:35.366738 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 12:03:35.366979 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":363,"threads_started":1}
W20260809 12:03:35.368021 3737 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46716
W20260809 12:03:35.368602 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 3541: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:03:35.368748 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 3541
I20260809 12:03:35.412901 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 2997
I20260809 12:03:35.414104 3087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:35.560262 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 2997
2026-08-09T12:03:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8951 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:03:35 dist-test-slave-sb2z krb5kdc[4012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:03:35 dist-test-slave-sb2z krb5kdc[4012](info): set up 2 sockets
Aug 09 12:03:35 dist-test-slave-sb2z krb5kdc[4012](info): commencing operation
krb5kdc: starting...
W20260809 12:03:36.877363 589 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:03:37 dist-test-slave-sb2z krb5kdc[4012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:03:37.100811 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:42591
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:42591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:37.431039 4028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:37.431365 4028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:37.431555 4028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:37.438676 4028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:37.438830 4028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:37.438952 4028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:37.439051 4028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:37.439142 4028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:37.448854 4028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:42591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:42591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:37.450178 4028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:37.451512 4028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:37.468067 4036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:37.468116 4039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:37.468025 4037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:37.468808 4028 server_base.cc:1034] running on GCE node
I20260809 12:03:37.469655 4028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:37.471050 4028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:37.472288 4028 hybrid_clock.cc:648] HybridClock initialized: now 1786277017472248 us; error 35 us; skew 500 ppm
I20260809 12:03:37.472641 4028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:37 dist-test-slave-sb2z krb5kdc[4012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:37.475857 4028 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:03:37.477360 4028 webserver.cc:460] Webserver started at http://127.0.147.126:37417/ using document root <none> and password file <none>
I20260809 12:03:37.478040 4028 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:37.478196 4028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:37.478506 4028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:37.481496 4028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "7d02eb3382de4ec5a5e061184404cc4e"
format_stamp: "Formatted at 2026-08-09 12:03:37 on dist-test-slave-sb2z"
server_key: "61f106dff170d9924153591d6752e522"
server_key_iv: "5141bc8c1ccb0cb62555e4e005c8cc07"
server_key_version: "encryptionkey@0"
I20260809 12:03:37.482304 4028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "7d02eb3382de4ec5a5e061184404cc4e"
format_stamp: "Formatted at 2026-08-09 12:03:37 on dist-test-slave-sb2z"
server_key: "61f106dff170d9924153591d6752e522"
server_key_iv: "5141bc8c1ccb0cb62555e4e005c8cc07"
server_key_version: "encryptionkey@0"
I20260809 12:03:37.487421 4028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:03:37.490960 4048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:37.492278 4028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:37.492462 4028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "7d02eb3382de4ec5a5e061184404cc4e"
format_stamp: "Formatted at 2026-08-09 12:03:37 on dist-test-slave-sb2z"
server_key: "61f106dff170d9924153591d6752e522"
server_key_iv: "5141bc8c1ccb0cb62555e4e005c8cc07"
server_key_version: "encryptionkey@0"
I20260809 12:03:37.492653 4028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:37.519430 4028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:37.523573 4028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:37.523890 4028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:37.542953 4028 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:42591
I20260809 12:03:37.542979 4118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:42591 every 8 connection(s)
I20260809 12:03:37.544443 4028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:37.548149 4119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:37.552398 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 4028
I20260809 12:03:37.552650 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:37.553493 589 external_mini_cluster.cc:1428] Setting key 4bdb2cf5db5af3b86b7973374d78cf08
I20260809 12:03:37.560072 4119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e: Bootstrap starting.
I20260809 12:03:37.563733 4119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:37.564927 4119 log.cc:824] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:37.568466 4119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e: No bootstrap required, opened a new log
Aug 09 12:03:37 dist-test-slave-sb2z krb5kdc[4012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:37.576239 4119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d02eb3382de4ec5a5e061184404cc4e" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42591 } }
I20260809 12:03:37.576625 4119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:37.576748 4119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d02eb3382de4ec5a5e061184404cc4e, State: Initialized, Role: FOLLOWER
I20260809 12:03:37.577399 4119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d02eb3382de4ec5a5e061184404cc4e" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42591 } }
I20260809 12:03:37.577687 4119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:37.577831 4119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:37.578037 4119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:37.578799 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:37.555719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.0.147.126:42591)
0809 12:03:37.556530 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:37.556542 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:37.556595 (+ 53us) server_negotiation.cc:407] Connection header received
0809 12:03:37.558574 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:37.558595 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:37.559002 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:37.559331 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:37.561317 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:37.562664 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:37.564520 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:37.565325 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:37.565607 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:37.570568 (+ 4961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:37.570577 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:37.570591 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:37.570633 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:37.574168 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:37.574926 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:37.574934 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:37.574941 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:37.575132 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:37.575594 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:37.575599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:37.575601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:37.576127 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:37.576506 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:37.576940 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 12:03:37.577373 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":396,"threads_started":1}
I20260809 12:03:37.580719 4119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d02eb3382de4ec5a5e061184404cc4e" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42591 } }
I20260809 12:03:37.581243 4119 leader_election.cc:302] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d02eb3382de4ec5a5e061184404cc4e; no voters:
I20260809 12:03:37.581877 4119 leader_election.cc:288] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:37.582042 4125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:37.582800 4125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [term 1 LEADER]: Becoming Leader. State: Replica: 7d02eb3382de4ec5a5e061184404cc4e, State: Running, Role: LEADER
I20260809 12:03:37.583382 4125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d02eb3382de4ec5a5e061184404cc4e" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42591 } }
I20260809 12:03:37.584051 4119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:37.586551 4127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d02eb3382de4ec5a5e061184404cc4e. Latest consensus state: current_term: 1 leader_uuid: "7d02eb3382de4ec5a5e061184404cc4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d02eb3382de4ec5a5e061184404cc4e" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42591 } } }
I20260809 12:03:37.586702 4126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d02eb3382de4ec5a5e061184404cc4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d02eb3382de4ec5a5e061184404cc4e" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42591 } } }
I20260809 12:03:37.587035 4127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:37.587086 4126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:37.588898 4139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:37.594431 4139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:37.603451 4139 catalog_manager.cc:1347] Generated new cluster ID: 5f50cd221cad4499963f4bba8fd97332
I20260809 12:03:37.603613 4139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:37.650902 4139 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:37.652793 4139 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:37.660615 4139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d02eb3382de4ec5a5e061184404cc4e: Generated new TSK 0
I20260809 12:03:37.661576 4139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:03:37.721238 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:42591
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:38.053694 4158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:38.053970 4158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:38.054160 4158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:38.061628 4158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:38.061738 4158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:38.061950 4158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:03:38.071985 4158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:42591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:38.073321 4158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:38.074631 4158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:38.088534 4166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:38.088745 4168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:38.088850 4165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:38.089718 4158 server_base.cc:1034] running on GCE node
I20260809 12:03:38.090466 4158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:38.091450 4158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:38.092715 4158 hybrid_clock.cc:648] HybridClock initialized: now 1786277018092657 us; error 53 us; skew 500 ppm
I20260809 12:03:38.093050 4158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:38 dist-test-slave-sb2z krb5kdc[4012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:38.100212 4158 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:03:38.101747 4158 webserver.cc:460] Webserver started at http://127.0.147.65:33015/ using document root <none> and password file <none>
I20260809 12:03:38.102448 4158 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:38.102602 4158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:38.102929 4158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:38.105886 4158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "c0ca3e27af284077ac01e88d4fc8771a"
format_stamp: "Formatted at 2026-08-09 12:03:38 on dist-test-slave-sb2z"
server_key: "e82d753906805be7163037b442bc322d"
server_key_iv: "f340f0759942f063b8b7646d4a248eff"
server_key_version: "encryptionkey@0"
I20260809 12:03:38.106693 4158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "c0ca3e27af284077ac01e88d4fc8771a"
format_stamp: "Formatted at 2026-08-09 12:03:38 on dist-test-slave-sb2z"
server_key: "e82d753906805be7163037b442bc322d"
server_key_iv: "f340f0759942f063b8b7646d4a248eff"
server_key_version: "encryptionkey@0"
I20260809 12:03:38.111835 4158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:03:38.115245 4178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:38.116592 4158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:38.116776 4158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "c0ca3e27af284077ac01e88d4fc8771a"
format_stamp: "Formatted at 2026-08-09 12:03:38 on dist-test-slave-sb2z"
server_key: "e82d753906805be7163037b442bc322d"
server_key_iv: "f340f0759942f063b8b7646d4a248eff"
server_key_version: "encryptionkey@0"
I20260809 12:03:38.116992 4158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:38.134471 4158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:38.138450 4158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:38.138738 4158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:38.139765 4158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:38.141112 4158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:38.141185 4158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:38.141299 4158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:38.141345 4158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:38.174823 4158 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:41653
I20260809 12:03:38.174917 4342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:41653 every 8 connection(s)
I20260809 12:03:38.176213 4158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:03:38.183292 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 4158
I20260809 12:03:38.183637 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:03:38.184427 589 external_mini_cluster.cc:1428] Setting key c2075f132caa71cd3c1a1d9e68961807
Aug 09 12:03:38 dist-test-slave-sb2z krb5kdc[4012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:38.199031 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:38.178969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:53667 (local address 127.0.147.126:42591)
0809 12:03:38.179454 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:38.179458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:38.180664 (+ 1206us) server_negotiation.cc:407] Connection header received
0809 12:03:38.182545 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:38.182550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:38.182651 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:38.182799 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:38.184517 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:38.185516 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:38.187879 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:38.188280 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:38.188386 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:38.192652 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:38.192657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:38.192660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:38.192691 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:38.195530 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:38.196645 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:38.196651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:38.196655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:38.196795 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:38.197269 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:38.197275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:38.197278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:38.197546 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:38.197708 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:38.198421 (+ 713us) server_negotiation.cc:299] Negotiation successful
0809 12:03:38.198651 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":246,"threads_started":1}
I20260809 12:03:38.200085 4351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:38.179695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:42591 (local address 127.0.147.65:53667)
0809 12:03:38.180492 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:38.180536 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:38.182272 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:38.183029 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:38.183038 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:38.183749 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:38.184339 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:38.184356 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:38.185759 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:38.185764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:38.187729 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:38.187747 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:38.188535 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:38.188545 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:38.188882 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:38.189647 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:38.189688 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:38.192414 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:38.195748 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:38.195757 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:38.195779 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:38.196524 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:38.196914 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:38.196919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:38.196923 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:38.197126 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:38.197683 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:38.197691 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:38.198034 (+ 343us) client_negotiation.cc:769] Sending connection context
0809 12:03:38.198307 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 12:03:38.198843 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":333,"threads_started":1}
I20260809 12:03:38.201840 4344 heartbeater.cc:347] Connected to a master server at 127.0.147.126:42591
I20260809 12:03:38.202174 4344 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:38.202909 4344 heartbeater.cc:511] Master 127.0.147.126:42591 requested a full tablet report, sending...
I20260809 12:03:38.205008 4071 ts_manager.cc:194] Registered new tserver with Master: c0ca3e27af284077ac01e88d4fc8771a (127.0.147.65:41653)
I20260809 12:03:38.206480 4071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:53667
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:03:38.238883 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:42591
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:38.568673 4355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:38.568928 4355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:38.569118 4355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:38.576995 4355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:38.577118 4355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:38.577314 4355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:03:38.587116 4355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:42591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:38.588429 4355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:38.589725 4355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:38.602810 4363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:38.602816 4365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:38.602842 4362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:38.603717 4355 server_base.cc:1034] running on GCE node
I20260809 12:03:38.604270 4355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:38.605103 4355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:38.606302 4355 hybrid_clock.cc:648] HybridClock initialized: now 1786277018606257 us; error 42 us; skew 500 ppm
I20260809 12:03:38.606623 4355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:38 dist-test-slave-sb2z krb5kdc[4012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:38.609874 4355 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:03:38.611207 4355 webserver.cc:460] Webserver started at http://127.0.147.66:46701/ using document root <none> and password file <none>
I20260809 12:03:38.611882 4355 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:38.611985 4355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:38.612249 4355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:38.615042 4355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "6e480c51ce23410aa33055f4e8da07bd"
format_stamp: "Formatted at 2026-08-09 12:03:38 on dist-test-slave-sb2z"
server_key: "290e8a7abcaf9a68864926d34debe110"
server_key_iv: "dabc78c368f860e11c4e618b63ae5318"
server_key_version: "encryptionkey@0"
I20260809 12:03:38.615922 4355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "6e480c51ce23410aa33055f4e8da07bd"
format_stamp: "Formatted at 2026-08-09 12:03:38 on dist-test-slave-sb2z"
server_key: "290e8a7abcaf9a68864926d34debe110"
server_key_iv: "dabc78c368f860e11c4e618b63ae5318"
server_key_version: "encryptionkey@0"
I20260809 12:03:38.620812 4355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:03:38.624349 4376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:38.625624 4355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:03:38.625788 4355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "6e480c51ce23410aa33055f4e8da07bd"
format_stamp: "Formatted at 2026-08-09 12:03:38 on dist-test-slave-sb2z"
server_key: "290e8a7abcaf9a68864926d34debe110"
server_key_iv: "dabc78c368f860e11c4e618b63ae5318"
server_key_version: "encryptionkey@0"
I20260809 12:03:38.625977 4355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:38.639362 4355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:38.643503 4355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:38.643848 4355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:38.644733 4355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:38.646010 4355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:38.646090 4355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:38.646253 4355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:38.646309 4355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:38.680846 4355 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:34731
I20260809 12:03:38.680909 4553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:34731 every 8 connection(s)
I20260809 12:03:38.682163 4355 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:03:38.690183 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 4355
I20260809 12:03:38.690408 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:03:38.690920 589 external_mini_cluster.cc:1428] Setting key 0324a0509685b042ac630cf967c1cb3a
Aug 09 12:03:38 dist-test-slave-sb2z krb5kdc[4012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:38.704154 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:38.684871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:53325 (local address 127.0.147.126:42591)
0809 12:03:38.685057 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:38.685063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:38.686219 (+ 1156us) server_negotiation.cc:407] Connection header received
0809 12:03:38.688135 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:38.688140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:38.688235 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:38.688349 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:38.689964 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:38.690861 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:38.692933 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:38.693363 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:38.693500 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:38.697075 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:38.697085 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:38.697093 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:38.697142 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:38.700889 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:38.701710 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:38.701718 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:38.701724 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:38.701888 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:38.702457 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:38.702460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:38.702462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:38.702723 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:38.702834 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:38.703735 (+ 901us) server_negotiation.cc:299] Negotiation successful
0809 12:03:38.703902 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:03:38.705618 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:38.685308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:42591 (local address 127.0.147.66:53325)
0809 12:03:38.686047 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:38.686098 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:38.687857 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:38.688577 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:38.688585 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:38.689221 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:38.689796 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:38.689813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:38.691026 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:38.691030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:38.692794 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:38.692806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:38.693585 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:38.693592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:38.693817 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:38.694488 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:38.694510 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:38.696822 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:38.701065 (+ 4243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:38.701076 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:38.701095 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:38.701576 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:38.702012 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:38.702020 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:38.702026 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:38.702297 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:38.702853 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:38.702863 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:38.703307 (+ 444us) client_negotiation.cc:769] Sending connection context
0809 12:03:38.703645 (+ 338us) client_negotiation.cc:240] Negotiation successful
0809 12:03:38.704196 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":322,"threads_started":1}
I20260809 12:03:38.707134 4554 heartbeater.cc:347] Connected to a master server at 127.0.147.126:42591
I20260809 12:03:38.707451 4554 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:38.708182 4554 heartbeater.cc:511] Master 127.0.147.126:42591 requested a full tablet report, sending...
I20260809 12:03:38.709688 4071 ts_manager.cc:194] Registered new tserver with Master: 6e480c51ce23410aa33055f4e8da07bd (127.0.147.66:34731)
I20260809 12:03:38.710932 4071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:53325
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:03:38.746363 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:42591
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:39.093647 4561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:39.093930 4561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:39.094138 4561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:39.101403 4561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:39.101511 4561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:39.101716 4561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:03:39.111409 4561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:42591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:39.112795 4561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:39.114117 4561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:39.127578 4567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:39.127589 4566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:39.127583 4571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:39.128391 4561 server_base.cc:1034] running on GCE node
I20260809 12:03:39.129079 4561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:39.129989 4561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:39.131273 4561 hybrid_clock.cc:648] HybridClock initialized: now 1786277019131220 us; error 36 us; skew 500 ppm
I20260809 12:03:39.131639 4561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:39 dist-test-slave-sb2z krb5kdc[4012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:39.134873 4561 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:03:39.136430 4561 webserver.cc:460] Webserver started at http://127.0.147.67:37015/ using document root <none> and password file <none>
I20260809 12:03:39.137111 4561 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:39.137264 4561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:39.137591 4561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:39.140573 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "47b2c449bd8b40289f8ae3c4e0e3a267"
format_stamp: "Formatted at 2026-08-09 12:03:39 on dist-test-slave-sb2z"
server_key: "d30c5c086204653ba34443b632e30c19"
server_key_iv: "c162b78b4406d63c47c3ab64bde1f8ae"
server_key_version: "encryptionkey@0"
I20260809 12:03:39.141379 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "47b2c449bd8b40289f8ae3c4e0e3a267"
format_stamp: "Formatted at 2026-08-09 12:03:39 on dist-test-slave-sb2z"
server_key: "d30c5c086204653ba34443b632e30c19"
server_key_iv: "c162b78b4406d63c47c3ab64bde1f8ae"
server_key_version: "encryptionkey@0"
I20260809 12:03:39.146509 4561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:03:39.149900 4581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:39.151216 4561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:03:39.151422 4561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "47b2c449bd8b40289f8ae3c4e0e3a267"
format_stamp: "Formatted at 2026-08-09 12:03:39 on dist-test-slave-sb2z"
server_key: "d30c5c086204653ba34443b632e30c19"
server_key_iv: "c162b78b4406d63c47c3ab64bde1f8ae"
server_key_version: "encryptionkey@0"
I20260809 12:03:39.151710 4561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:39.168603 4561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:39.172508 4561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:39.172798 4561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:39.173799 4561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:39.175175 4561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:39.175283 4561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:39.175439 4561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:39.175515 4561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:39.209575 4344 heartbeater.cc:503] Master 127.0.147.126:42591 was elected leader, sending a full tablet report...
I20260809 12:03:39.211529 4561 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:43933
I20260809 12:03:39.211578 4742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:43933 every 8 connection(s)
I20260809 12:03:39.212858 4561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:03:39.217854 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 4561
I20260809 12:03:39.218086 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:03:39.218525 589 external_mini_cluster.cc:1428] Setting key f9267622482e4f11896e699c18c92633
Aug 09 12:03:39 dist-test-slave-sb2z krb5kdc[4012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:39.233484 4745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:39.215481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:38457 (local address 127.0.147.126:42591)
0809 12:03:39.215926 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:39.215931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:39.216992 (+ 1061us) server_negotiation.cc:407] Connection header received
0809 12:03:39.218954 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:39.218960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:39.219049 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:39.219168 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:39.220860 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:39.221953 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:39.223450 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:39.223959 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:39.224059 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:39.227330 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:39.227335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:39.227339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:39.227370 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:39.230110 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:39.230838 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:39.230842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:39.230844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:39.231003 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:39.231414 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:39.231419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:39.231421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:39.231804 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:39.231953 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:39.232854 (+ 901us) server_negotiation.cc:299] Negotiation successful
0809 12:03:39.233120 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":226,"threads_started":1}
I20260809 12:03:39.234515 4746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:39.216110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:42591 (local address 127.0.147.67:38457)
0809 12:03:39.216826 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:39.216862 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:39.218697 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:39.219432 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:39.219444 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:39.220072 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:39.220673 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:39.220704 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:39.222124 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:39.222128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:39.223309 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:39.223322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:39.224171 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:39.224176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:39.224358 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:39.225000 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:39.225034 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:39.227138 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:39.230232 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:39.230238 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:39.230250 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:39.230733 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:39.231101 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:39.231104 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:39.231106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:39.231277 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:39.231949 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:39.231960 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:39.232387 (+ 427us) client_negotiation.cc:769] Sending connection context
0809 12:03:39.232703 (+ 316us) client_negotiation.cc:240] Negotiation successful
0809 12:03:39.233231 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":282,"threads_started":1}
I20260809 12:03:39.235882 4743 heartbeater.cc:347] Connected to a master server at 127.0.147.126:42591
I20260809 12:03:39.236141 4743 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:39.236792 4743 heartbeater.cc:511] Master 127.0.147.126:42591 requested a full tablet report, sending...
I20260809 12:03:39.238163 4071 ts_manager.cc:194] Registered new tserver with Master: 47b2c449bd8b40289f8ae3c4e0e3a267 (127.0.147.67:43933)
I20260809 12:03:39.238982 4071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:38457
I20260809 12:03:39.246254 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 12:03:39.288077 4745 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:03:39.285307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55406 (local address 127.0.147.126:42591)
0809 12:03:39.285527 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:39.285535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:39.286092 (+ 557us) server_negotiation.cc:407] Connection header received
0809 12:03:39.286231 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:39.286236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:39.286311 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:39.286418 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:39.287781 (+ 1363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55406: BlockingRecv error: recv got EOF from 127.0.0.1:55406 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:03:39.291721 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 4158
W20260809 12:03:39.294680 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 4158: Not authorized: Client connection negotiation failed: client connection to 127.0.147.65:41653: server requires authentication, but client does not have Kerberos credentials available
I20260809 12:03:39.294891 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 4158
I20260809 12:03:39.320866 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 4355
W20260809 12:03:39.324465 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 4355: Not authorized: Client connection negotiation failed: client connection to 127.0.147.66:34731: server requires authentication, but client does not have Kerberos credentials available
I20260809 12:03:39.324644 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 4355
I20260809 12:03:39.355111 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 4561
W20260809 12:03:39.358229 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 4561: Not authorized: Client connection negotiation failed: client connection to 127.0.147.67:43933: server requires authentication, but client does not have Kerberos credentials available
I20260809 12:03:39.358381 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 4561
I20260809 12:03:39.382294 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 4028
I20260809 12:03:39.383667 4114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:39.488317 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 4028
2026-08-09T12:03:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3924 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:03:39 dist-test-slave-sb2z krb5kdc[4767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:03:39 dist-test-slave-sb2z krb5kdc[4767](info): set up 2 sockets
Aug 09 12:03:39 dist-test-slave-sb2z krb5kdc[4767](info): commencing operation
krb5kdc: starting...
W20260809 12:03:40.779078 589 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:03:40 dist-test-slave-sb2z krb5kdc[4767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:03:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:03:41.010535 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:33529
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:33529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:41.345129 4785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:41.345413 4785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:41.345605 4785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:41.352788 4785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:41.352893 4785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:41.353006 4785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:41.353112 4785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:41.353201 4785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:41.362890 4785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:33529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:33529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:41.364239 4785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:41.365597 4785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:41.377990 4793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:41.378047 4798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:41.378073 4794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:41.378895 4785 server_base.cc:1034] running on GCE node
I20260809 12:03:41.379839 4785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:41.381271 4785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:41.382488 4785 hybrid_clock.cc:648] HybridClock initialized: now 1786277021382479 us; error 72 us; skew 500 ppm
I20260809 12:03:41.382846 4785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:41 dist-test-slave-sb2z krb5kdc[4767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:41.386051 4785 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:03:41.387621 4785 webserver.cc:460] Webserver started at http://127.0.147.126:38449/ using document root <none> and password file <none>
I20260809 12:03:41.388317 4785 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:41.388455 4785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:41.388772 4785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:41.391830 4785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "6b2c1646591c4bd8bc74216a337928d8"
format_stamp: "Formatted at 2026-08-09 12:03:41 on dist-test-slave-sb2z"
server_key: "291e397b071ec0c25b70cf2fb35d4608"
server_key_iv: "686a60aaa69d4bf2775c58d3d232ed13"
server_key_version: "encryptionkey@0"
I20260809 12:03:41.392642 4785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "6b2c1646591c4bd8bc74216a337928d8"
format_stamp: "Formatted at 2026-08-09 12:03:41 on dist-test-slave-sb2z"
server_key: "291e397b071ec0c25b70cf2fb35d4608"
server_key_iv: "686a60aaa69d4bf2775c58d3d232ed13"
server_key_version: "encryptionkey@0"
I20260809 12:03:41.397770 4785 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:03:41.401142 4808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:41.402376 4785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:03:41.402559 4785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "6b2c1646591c4bd8bc74216a337928d8"
format_stamp: "Formatted at 2026-08-09 12:03:41 on dist-test-slave-sb2z"
server_key: "291e397b071ec0c25b70cf2fb35d4608"
server_key_iv: "686a60aaa69d4bf2775c58d3d232ed13"
server_key_version: "encryptionkey@0"
I20260809 12:03:41.402765 4785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:41.414664 4785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:41.418514 4785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:41.418802 4785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:41.437849 4785 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:33529
I20260809 12:03:41.437852 4885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:33529 every 8 connection(s)
I20260809 12:03:41.439344 4785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:41.441012 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 4785
I20260809 12:03:41.441237 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:41.442085 589 external_mini_cluster.cc:1428] Setting key 033413512d34eae8715ae50599776c22
I20260809 12:03:41.444180 4887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:03:41 dist-test-slave-sb2z krb5kdc[4767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:41.458041 4887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8: Bootstrap starting.
I20260809 12:03:41.462473 4887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:41.463817 4887 log.cc:824] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:41.467056 4892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:41.444040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.0.147.126:33529)
0809 12:03:41.444931 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:41.444942 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:41.444983 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:03:41.446622 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:41.446656 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:41.447023 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:41.447338 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:41.449285 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:41.450668 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:41.452431 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:41.453076 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:41.453208 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:41.458104 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:41.458117 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:41.458137 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:41.458190 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:41.462244 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:41.463037 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:41.463045 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:41.463053 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:41.463267 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:41.463704 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:41.463709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:41.463712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:41.464474 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:41.464737 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:41.465295 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 12:03:41.465758 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":466,"threads_started":1}
I20260809 12:03:41.467198 4887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8: No bootstrap required, opened a new log
I20260809 12:03:41.471144 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:03:41.475159 4887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2c1646591c4bd8bc74216a337928d8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33529 } }
I20260809 12:03:41.475577 4887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:41.475703 4887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b2c1646591c4bd8bc74216a337928d8, State: Initialized, Role: FOLLOWER
I20260809 12:03:41.476250 4887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2c1646591c4bd8bc74216a337928d8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33529 } }
I20260809 12:03:41.476506 4887 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:41.476655 4887 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:41.476841 4887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:41.479141 4887 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2c1646591c4bd8bc74216a337928d8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33529 } }
I20260809 12:03:41.479689 4887 leader_election.cc:302] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b2c1646591c4bd8bc74216a337928d8; no voters:
I20260809 12:03:41.480347 4887 leader_election.cc:288] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:41.480453 4894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:41.482928 4887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:41.483806 4894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [term 1 LEADER]: Becoming Leader. State: Replica: 6b2c1646591c4bd8bc74216a337928d8, State: Running, Role: LEADER
I20260809 12:03:41.484449 4894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2c1646591c4bd8bc74216a337928d8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33529 } }
I20260809 12:03:41.488078 4903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b2c1646591c4bd8bc74216a337928d8. Latest consensus state: current_term: 1 leader_uuid: "6b2c1646591c4bd8bc74216a337928d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2c1646591c4bd8bc74216a337928d8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33529 } } }
I20260809 12:03:41.488699 4903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:41.487970 4898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b2c1646591c4bd8bc74216a337928d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2c1646591c4bd8bc74216a337928d8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33529 } } }
I20260809 12:03:41.489300 4898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:41.492201 4911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:41.497151 4911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:41.512931 4911 catalog_manager.cc:1347] Generated new cluster ID: 07fcec47264a475290402b8cc0cdffbf
I20260809 12:03:41.513118 4911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:41.523514 4911 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:41.525584 4911 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:41.533252 4892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:41.524130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.0.147.126:33529)
0809 12:03:41.524306 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:41.524313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:41.524342 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:03:41.524622 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:41.524628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:41.524706 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:41.524777 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:41.526175 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:41.527360 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:41.529236 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:41.529941 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:41.530084 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:41.531249 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:41.531255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:41.531260 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:41.532389 (+ 1129us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:41.532443 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:41.532479 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:41.532601 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:41.532759 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 12:03:41.532867 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:03:41.537757 4911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b2c1646591c4bd8bc74216a337928d8: Generated new TSK 0
I20260809 12:03:41.538664 4911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:03:41.551095 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 4785
I20260809 12:03:41.552620 4881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:41.643563 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 4785
2026-08-09T12:03:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2147 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:03:41 dist-test-slave-sb2z krb5kdc[4935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:03:41 dist-test-slave-sb2z krb5kdc[4935](info): set up 2 sockets
Aug 09 12:03:41 dist-test-slave-sb2z krb5kdc[4935](info): commencing operation
krb5kdc: starting...
W20260809 12:03:42.922612 589 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:03:43 dist-test-slave-sb2z krb5kdc[4935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:03:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:03:43.151671 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:40667
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:40667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:43.506829 4951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:43.507167 4951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:43.507323 4951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:43.514578 4951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:43.514719 4951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:43.514824 4951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:43.514927 4951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:43.515021 4951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:43.524911 4951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:40667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:40667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:43.526302 4951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:43.527734 4951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:43.539568 4961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:43.539661 4958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:43.539645 4957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:43.545953 4951 server_base.cc:1034] running on GCE node
I20260809 12:03:43.547194 4951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:43.549353 4951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:43.550647 4951 hybrid_clock.cc:648] HybridClock initialized: now 1786277023550563 us; error 65 us; skew 500 ppm
I20260809 12:03:43.551186 4951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:43 dist-test-slave-sb2z krb5kdc[4935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277023, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:43.557888 4951 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:03:43.559501 4951 webserver.cc:460] Webserver started at http://127.0.147.126:33127/ using document root <none> and password file <none>
I20260809 12:03:43.560272 4951 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:43.560425 4951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:43.560751 4951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:43.564014 4951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "85138b9b8bf84bab8a3981ff9b2d2869"
format_stamp: "Formatted at 2026-08-09 12:03:43 on dist-test-slave-sb2z"
server_key: "b679d9caf5c6112e7c5994b3a1ebecc0"
server_key_iv: "63e0b8f833019fa7ca3370bd71be4cf0"
server_key_version: "encryptionkey@0"
I20260809 12:03:43.564801 4951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "85138b9b8bf84bab8a3981ff9b2d2869"
format_stamp: "Formatted at 2026-08-09 12:03:43 on dist-test-slave-sb2z"
server_key: "b679d9caf5c6112e7c5994b3a1ebecc0"
server_key_iv: "63e0b8f833019fa7ca3370bd71be4cf0"
server_key_version: "encryptionkey@0"
I20260809 12:03:43.569940 4951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 12:03:43.573443 4976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:43.574839 4951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:03:43.575062 4951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "85138b9b8bf84bab8a3981ff9b2d2869"
format_stamp: "Formatted at 2026-08-09 12:03:43 on dist-test-slave-sb2z"
server_key: "b679d9caf5c6112e7c5994b3a1ebecc0"
server_key_iv: "63e0b8f833019fa7ca3370bd71be4cf0"
server_key_version: "encryptionkey@0"
I20260809 12:03:43.575286 4951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:43.596298 4951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:43.601336 4951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:43.601650 4951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:43.620975 4951 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:40667
I20260809 12:03:43.620999 5043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:40667 every 8 connection(s)
I20260809 12:03:43.622387 4951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:43.625007 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 4951
I20260809 12:03:43.625249 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:43.626078 589 external_mini_cluster.cc:1428] Setting key 9c53f3e0dfec3b045673be998bc1c6ea
I20260809 12:03:43.626763 5047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:43.638872 5047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869: Bootstrap starting.
Aug 09 12:03:43 dist-test-slave-sb2z krb5kdc[4935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:43.642505 5047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:43.643656 5047 log.cc:824] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:43.646889 5047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869: No bootstrap required, opened a new log
I20260809 12:03:43.649690 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:43.628214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60852 (local address 127.0.147.126:40667)
0809 12:03:43.628949 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:43.628979 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:43.629019 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:03:43.631326 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:43.631348 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:43.631784 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:43.632190 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:43.633634 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:43.634895 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:43.636228 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:43.636970 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:43.637228 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:43.641852 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:43.641859 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:43.641874 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:43.641909 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:43.645482 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:43.646311 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:43.646318 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:43.646324 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:43.646488 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:43.646908 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:43.646911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:43.646913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:43.647477 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:43.647827 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:43.648265 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 12:03:43.648596 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":339,"threads_started":1}
I20260809 12:03:43.655006 5047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85138b9b8bf84bab8a3981ff9b2d2869" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40667 } }
I20260809 12:03:43.655341 5047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:43.655462 5047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85138b9b8bf84bab8a3981ff9b2d2869, State: Initialized, Role: FOLLOWER
I20260809 12:03:43.656035 5047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85138b9b8bf84bab8a3981ff9b2d2869" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40667 } }
I20260809 12:03:43.656286 5047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:43.656440 5047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:43.656626 5047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:43.658783 5047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85138b9b8bf84bab8a3981ff9b2d2869" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40667 } }
I20260809 12:03:43.659188 5047 leader_election.cc:302] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85138b9b8bf84bab8a3981ff9b2d2869; no voters:
I20260809 12:03:43.659909 5047 leader_election.cc:288] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:43.660034 5057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:43.660759 5057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [term 1 LEADER]: Becoming Leader. State: Replica: 85138b9b8bf84bab8a3981ff9b2d2869, State: Running, Role: LEADER
I20260809 12:03:43.661258 5057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85138b9b8bf84bab8a3981ff9b2d2869" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40667 } }
I20260809 12:03:43.661917 5047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:43.664254 5059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85138b9b8bf84bab8a3981ff9b2d2869. Latest consensus state: current_term: 1 leader_uuid: "85138b9b8bf84bab8a3981ff9b2d2869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85138b9b8bf84bab8a3981ff9b2d2869" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40667 } } }
I20260809 12:03:43.664662 5059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:43.664645 5058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85138b9b8bf84bab8a3981ff9b2d2869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85138b9b8bf84bab8a3981ff9b2d2869" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40667 } } }
I20260809 12:03:43.664978 5058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:43.665567 5068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:43.671106 5068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:43.680927 5068 catalog_manager.cc:1347] Generated new cluster ID: 6ba6bb8add3d49a4b74b18fb2e3ffcc9
I20260809 12:03:43.681046 5068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:43.698459 5068 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:43.699819 5068 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:43.706773 5068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85138b9b8bf84bab8a3981ff9b2d2869: Generated new TSK 0
I20260809 12:03:43.707780 5068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:03:43.775049 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40667
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:44.115053 5085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:44.115352 5085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:44.115448 5085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:44.122694 5085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:44.122792 5085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:44.122902 5085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:03:44.132261 5085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:44.133419 5085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:44.134680 5085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:44.148082 5092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:44.148087 5094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:44.148010 5096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:44.148929 5085 server_base.cc:1034] running on GCE node
I20260809 12:03:44.149605 5085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:44.150593 5085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:44.151808 5085 hybrid_clock.cc:648] HybridClock initialized: now 1786277024151750 us; error 48 us; skew 500 ppm
I20260809 12:03:44.152102 5085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:44 dist-test-slave-sb2z krb5kdc[4935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:44.155267 5085 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:03:44.156790 5085 webserver.cc:460] Webserver started at http://127.0.147.65:36139/ using document root <none> and password file <none>
I20260809 12:03:44.157433 5085 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:44.157534 5085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:44.157792 5085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:44.160778 5085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "390dff3e4070408398cc4f38128ab2b7"
format_stamp: "Formatted at 2026-08-09 12:03:44 on dist-test-slave-sb2z"
server_key: "6f3900a79e6c9d7b668b49329e09997a"
server_key_iv: "c88592d0b677160e4981d8dbad2f03ca"
server_key_version: "encryptionkey@0"
I20260809 12:03:44.161546 5085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "390dff3e4070408398cc4f38128ab2b7"
format_stamp: "Formatted at 2026-08-09 12:03:44 on dist-test-slave-sb2z"
server_key: "6f3900a79e6c9d7b668b49329e09997a"
server_key_iv: "c88592d0b677160e4981d8dbad2f03ca"
server_key_version: "encryptionkey@0"
I20260809 12:03:44.166973 5085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:03:44.170589 5109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:44.171945 5085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:44.172138 5085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "390dff3e4070408398cc4f38128ab2b7"
format_stamp: "Formatted at 2026-08-09 12:03:44 on dist-test-slave-sb2z"
server_key: "6f3900a79e6c9d7b668b49329e09997a"
server_key_iv: "c88592d0b677160e4981d8dbad2f03ca"
server_key_version: "encryptionkey@0"
I20260809 12:03:44.172338 5085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:44.200953 5085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:44.204964 5085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:44.205282 5085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:44.206328 5085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:44.207847 5085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:44.207922 5085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:44.208010 5085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:44.208072 5085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:44.242830 5085 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:38941
I20260809 12:03:44.242875 5281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:38941 every 8 connection(s)
I20260809 12:03:44.244170 5085 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:03:44.249181 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 5085
I20260809 12:03:44.249397 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:03:44.250046 589 external_mini_cluster.cc:1428] Setting key 45132a8db446b7514ca16318b423b350
Aug 09 12:03:44 dist-test-slave-sb2z krb5kdc[4935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:44.264811 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:44.246742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:40785 (local address 127.0.147.126:40667)
0809 12:03:44.247163 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:44.247168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:44.248189 (+ 1021us) server_negotiation.cc:407] Connection header received
0809 12:03:44.250097 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:44.250103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:44.250193 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:44.250313 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:44.252329 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:44.253104 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:44.254664 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:44.255075 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:44.255209 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:44.258926 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:44.258930 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:44.258934 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:44.258964 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:44.261573 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:44.262384 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:44.262389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:44.262398 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:44.262518 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:44.263025 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:44.263031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:44.263034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:44.263351 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:44.263503 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:44.264221 (+ 718us) server_negotiation.cc:299] Negotiation successful
0809 12:03:44.264445 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":206,"threads_started":1}
I20260809 12:03:44.265727 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:44.247289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40667 (local address 127.0.147.65:40785)
0809 12:03:44.248026 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:44.248073 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:44.249826 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:44.250556 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:44.250587 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:44.251320 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:44.252141 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:44.252168 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:44.253260 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:44.253265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:44.254499 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:44.254512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:44.255315 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:44.255323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:44.255644 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:44.256321 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:44.256344 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:44.258721 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:44.261732 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:44.261740 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:44.261755 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:44.262261 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:44.262630 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:44.262635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:44.262639 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:44.262896 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:44.263478 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:44.263485 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:44.263848 (+ 363us) client_negotiation.cc:769] Sending connection context
0809 12:03:44.264103 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 12:03:44.264611 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":339,"threads_started":1}
I20260809 12:03:44.267279 5283 heartbeater.cc:347] Connected to a master server at 127.0.147.126:40667
I20260809 12:03:44.267642 5283 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:44.268388 5283 heartbeater.cc:511] Master 127.0.147.126:40667 requested a full tablet report, sending...
I20260809 12:03:44.270809 4996 ts_manager.cc:194] Registered new tserver with Master: 390dff3e4070408398cc4f38128ab2b7 (127.0.147.65:38941)
I20260809 12:03:44.273044 4996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:40785
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:03:44.303165 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40667
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:44.631376 5295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:44.631699 5295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:44.631901 5295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:44.639045 5295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:44.639145 5295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:44.639330 5295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:03:44.648941 5295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:44.650151 5295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:44.651445 5295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:44.664439 5302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:44.664691 5306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:44.665161 5303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:44.665819 5295 server_base.cc:1034] running on GCE node
I20260809 12:03:44.666414 5295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:44.667367 5295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:44.668569 5295 hybrid_clock.cc:648] HybridClock initialized: now 1786277024668495 us; error 73 us; skew 500 ppm
I20260809 12:03:44.668838 5295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:44 dist-test-slave-sb2z krb5kdc[4935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:44.671993 5295 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:03:44.673439 5295 webserver.cc:460] Webserver started at http://127.0.147.66:33177/ using document root <none> and password file <none>
I20260809 12:03:44.674115 5295 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:44.674263 5295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:44.674574 5295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:44.677565 5295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "fe9d7b9acf4a4ee588acbd4b9529d578"
format_stamp: "Formatted at 2026-08-09 12:03:44 on dist-test-slave-sb2z"
server_key: "1042367cd7017ea14543618e244fc953"
server_key_iv: "13d3b0578e2d4f659095d6349f604185"
server_key_version: "encryptionkey@0"
I20260809 12:03:44.678349 5295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "fe9d7b9acf4a4ee588acbd4b9529d578"
format_stamp: "Formatted at 2026-08-09 12:03:44 on dist-test-slave-sb2z"
server_key: "1042367cd7017ea14543618e244fc953"
server_key_iv: "13d3b0578e2d4f659095d6349f604185"
server_key_version: "encryptionkey@0"
I20260809 12:03:44.683609 5295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:03:44.687041 5315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:44.688380 5295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:03:44.688558 5295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "fe9d7b9acf4a4ee588acbd4b9529d578"
format_stamp: "Formatted at 2026-08-09 12:03:44 on dist-test-slave-sb2z"
server_key: "1042367cd7017ea14543618e244fc953"
server_key_iv: "13d3b0578e2d4f659095d6349f604185"
server_key_version: "encryptionkey@0"
I20260809 12:03:44.688753 5295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:44.715381 5295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:44.719921 5295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:44.720238 5295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:44.721227 5295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:44.722645 5295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:44.722755 5295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:44.722908 5295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:44.722983 5295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:44.757819 5295 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:40149
I20260809 12:03:44.757917 5484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:40149 every 8 connection(s)
I20260809 12:03:44.759135 5295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:03:44.764995 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 5295
I20260809 12:03:44.765195 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:03:44.765771 589 external_mini_cluster.cc:1428] Setting key 3a681c56fd2b548b6f694ba40e65e379
Aug 09 12:03:44 dist-test-slave-sb2z krb5kdc[4935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:44.782048 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:44.761863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:57685 (local address 127.0.147.126:40667)
0809 12:03:44.762074 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:44.762080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:44.763346 (+ 1266us) server_negotiation.cc:407] Connection header received
0809 12:03:44.765613 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:44.765618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:44.765688 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:44.765779 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:44.767667 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:44.768503 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:44.771106 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:44.771600 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:44.771707 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:44.776039 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:44.776044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:44.776047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:44.776096 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:44.778831 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:44.779599 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:44.779605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:44.779609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:44.779748 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:44.780310 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:44.780321 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:44.780323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:44.780593 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:44.780755 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:44.781439 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 12:03:44.781680 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:03:44.783025 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:44.762399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40667 (local address 127.0.147.66:57685)
0809 12:03:44.763146 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:44.763228 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:44.765310 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:44.766058 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:44.766069 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:44.766815 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:44.767456 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:44.767473 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:44.768677 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:44.768682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:44.770935 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:44.770959 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:44.771833 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:44.771843 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:44.772167 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:44.772899 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:44.772933 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:44.775767 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:44.778989 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:44.778998 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:44.779015 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:44.779444 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:44.779866 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:44.779871 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:44.779875 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:44.780181 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:44.780728 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:44.780735 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:44.781076 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 12:03:44.781341 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 12:03:44.781870 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":276,"threads_started":1}
I20260809 12:03:44.784667 5485 heartbeater.cc:347] Connected to a master server at 127.0.147.126:40667
I20260809 12:03:44.785096 5485 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:44.785836 5485 heartbeater.cc:511] Master 127.0.147.126:40667 requested a full tablet report, sending...
I20260809 12:03:44.787410 4996 ts_manager.cc:194] Registered new tserver with Master: fe9d7b9acf4a4ee588acbd4b9529d578 (127.0.147.66:40149)
I20260809 12:03:44.788254 4996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:57685
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:03:44.818851 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40667
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:45.152047 5490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:45.152308 5490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:45.152520 5490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:45.159740 5490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:45.159861 5490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:45.160046 5490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:03:45.169992 5490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:45.171221 5490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:45.172533 5490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:45.185770 5501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:45.185832 5498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:45.185806 5499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:45.186911 5490 server_base.cc:1034] running on GCE node
I20260809 12:03:45.187693 5490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:45.188593 5490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:45.189823 5490 hybrid_clock.cc:648] HybridClock initialized: now 1786277025189785 us; error 53 us; skew 500 ppm
I20260809 12:03:45.190142 5490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[4935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277025, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:45.193984 5490 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:03:45.195442 5490 webserver.cc:460] Webserver started at http://127.0.147.67:42629/ using document root <none> and password file <none>
I20260809 12:03:45.196177 5490 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:45.196310 5490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:45.196640 5490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:45.199712 5490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "414548c6909546aa89d0965cc3e93c2e"
format_stamp: "Formatted at 2026-08-09 12:03:45 on dist-test-slave-sb2z"
server_key: "75f615ebc4d310b28a7ca5f3486da653"
server_key_iv: "b86c2e75174f481e23ea35d4474856a1"
server_key_version: "encryptionkey@0"
I20260809 12:03:45.200532 5490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "414548c6909546aa89d0965cc3e93c2e"
format_stamp: "Formatted at 2026-08-09 12:03:45 on dist-test-slave-sb2z"
server_key: "75f615ebc4d310b28a7ca5f3486da653"
server_key_iv: "b86c2e75174f481e23ea35d4474856a1"
server_key_version: "encryptionkey@0"
I20260809 12:03:45.205749 5490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 12:03:45.209206 5511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:45.210477 5490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:03:45.210649 5490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "414548c6909546aa89d0965cc3e93c2e"
format_stamp: "Formatted at 2026-08-09 12:03:45 on dist-test-slave-sb2z"
server_key: "75f615ebc4d310b28a7ca5f3486da653"
server_key_iv: "b86c2e75174f481e23ea35d4474856a1"
server_key_version: "encryptionkey@0"
I20260809 12:03:45.210830 5490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:45.225016 5490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:45.228919 5490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:45.229230 5490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:45.230170 5490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:45.231498 5490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:45.231613 5490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:45.231791 5490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:45.231855 5490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:45.268095 5490 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:36965
I20260809 12:03:45.268112 5670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:36965 every 8 connection(s)
I20260809 12:03:45.269397 5490 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:03:45.276537 5283 heartbeater.cc:503] Master 127.0.147.126:40667 was elected leader, sending a full tablet report...
I20260809 12:03:45.279191 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 5490
I20260809 12:03:45.279381 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:03:45.279897 589 external_mini_cluster.cc:1428] Setting key 5fdc3fc1eef93a98a0568fd962478c79
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[4935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277025, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:45.290663 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:45.272079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:41053 (local address 127.0.147.126:40667)
0809 12:03:45.272306 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:45.272312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:45.273467 (+ 1155us) server_negotiation.cc:407] Connection header received
0809 12:03:45.275151 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:45.275155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:45.275249 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:45.275339 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:45.277152 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.278015 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.279897 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.280436 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.280582 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:45.284190 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:45.284195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:45.284198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:45.284230 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:45.287578 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.288322 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.288327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.288331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.288483 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.288924 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.288929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.288933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.289208 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:45.289384 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:45.290087 (+ 703us) server_negotiation.cc:299] Negotiation successful
0809 12:03:45.290301 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:03:45.291518 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:45.272609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40667 (local address 127.0.147.67:41053)
0809 12:03:45.273298 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:45.273344 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:45.274911 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:45.275587 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:45.275604 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:45.276166 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:45.276932 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:45.276952 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.278183 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.278196 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:45.279723 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:45.279736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.280685 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.280696 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:45.280951 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:45.281641 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:45.281663 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:45.283980 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:45.287729 (+ 3749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:45.287738 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:45.287752 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:45.288178 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:45.288600 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:45.288604 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:45.288608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:45.288800 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:45.289346 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:45.289356 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:45.289678 (+ 322us) client_negotiation.cc:769] Sending connection context
0809 12:03:45.289950 (+ 272us) client_negotiation.cc:240] Negotiation successful
0809 12:03:45.290476 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":271,"threads_started":1}
I20260809 12:03:45.292865 5671 heartbeater.cc:347] Connected to a master server at 127.0.147.126:40667
I20260809 12:03:45.293202 5671 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:45.294145 5671 heartbeater.cc:511] Master 127.0.147.126:40667 requested a full tablet report, sending...
I20260809 12:03:45.295604 4996 ts_manager.cc:194] Registered new tserver with Master: 414548c6909546aa89d0965cc3e93c2e (127.0.147.67:36965)
I20260809 12:03:45.296444 4996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:41053
I20260809 12:03:45.306984 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[4935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277025, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[4935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277025, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:45.366410 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:45.352714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.0.147.126:40667)
0809 12:03:45.352892 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:45.352897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:45.353309 (+ 412us) server_negotiation.cc:407] Connection header received
0809 12:03:45.353461 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:45.353465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:45.353533 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:45.353627 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:45.354963 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.355768 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.356932 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.357391 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.357490 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:45.361265 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:45.361270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:45.361273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:45.361304 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:45.364168 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.364767 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.364770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.364773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.364906 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.365267 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.365271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.365272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.365591 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:45.365770 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:45.366031 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 12:03:45.366188 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 12:03:45.367089 4996 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60860
I20260809 12:03:45.368862 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 5085
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[4935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277025, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:45.384223 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:45.370011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.0.147.65:38941)
0809 12:03:45.370594 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:45.370601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:45.370626 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:03:45.370679 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:45.370685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:45.370899 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:45.371032 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:45.372276 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.373272 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.374378 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.374920 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.375017 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:45.378966 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:45.378977 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:45.378990 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:45.379044 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:45.381883 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.382491 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.382498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.382500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.382628 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.382938 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.382941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.382943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.383252 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:45.383471 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:45.383757 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 12:03:45.383948 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":276,"threads_started":1}
W20260809 12:03:45.385448 5274 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52680
W20260809 12:03:45.386588 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 5085: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:03:45.386807 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 5085
I20260809 12:03:45.413223 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 5295
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[4935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277025, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:03:45.429020 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:45.414274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.0.147.66:40149)
0809 12:03:45.414882 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:45.414890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:45.414915 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:03:45.414978 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:45.414984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:45.415217 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:45.415342 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:45.416624 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.417627 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.418822 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.419384 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.419484 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:45.423676 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:45.423687 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:45.423700 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:45.423731 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:45.426403 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.427010 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.427017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.427019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.427134 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.427433 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.427436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.427438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.427905 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:45.428137 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:45.428414 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 12:03:45.428667 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":318,"threads_started":1}
W20260809 12:03:45.430312 5478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45002
W20260809 12:03:45.431347 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 5295: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:03:45.431507 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 5295
I20260809 12:03:45.461743 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 5490
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[4935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277025, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:03:45.478410 5690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:45.462683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53554 (local address 127.0.147.67:36965)
0809 12:03:45.463347 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:45.463354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:45.463382 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:03:45.463441 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:45.463446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:45.463730 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:45.463901 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:45.465319 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.466423 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.467894 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:45.468454 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:45.468565 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:45.472877 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:45.472889 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:45.472903 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:45.472936 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:45.475991 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.476571 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.476578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.476580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.476704 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:45.477047 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:45.477049 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:45.477051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:45.477366 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:45.477596 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:45.477914 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 12:03:45.478127 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":308,"threads_started":1}
W20260809 12:03:45.479811 5665 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53554
W20260809 12:03:45.481002 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 5490: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:03:45.481185 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 5490
I20260809 12:03:45.510648 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 4951
I20260809 12:03:45.511962 5038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:45.637179 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 4951
2026-08-09T12:03:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3995 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[5697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[5697](info): set up 2 sockets
Aug 09 12:03:45 dist-test-slave-sb2z krb5kdc[5697](info): commencing operation
krb5kdc: starting...
W20260809 12:03:46.924715 589 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:03:47 dist-test-slave-sb2z krb5kdc[5697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:03:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:03:47.153029 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45273
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:38157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:45273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:47.483055 5714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:47.483330 5714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:47.483554 5714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:47.490670 5714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:47.490792 5714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:47.490906 5714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:47.491017 5714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:47.491110 5714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:47.500928 5714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:38157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:45273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:47.502233 5714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:47.503593 5714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:47.515483 5725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:47.515723 5724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:47.515739 5727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:47.516750 5714 server_base.cc:1034] running on GCE node
I20260809 12:03:47.517479 5714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:47.518780 5714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:47.519979 5714 hybrid_clock.cc:648] HybridClock initialized: now 1786277027519923 us; error 53 us; skew 500 ppm
I20260809 12:03:47.520337 5714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:47 dist-test-slave-sb2z krb5kdc[5697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277027, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:47.523487 5714 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:03:47.524996 5714 webserver.cc:460] Webserver started at http://127.0.147.126:39665/ using document root <none> and password file <none>
I20260809 12:03:47.525681 5714 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:47.525786 5714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:47.526047 5714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:47.529050 5714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "f50dd95d153b4cbf83befe29e1cc6808"
format_stamp: "Formatted at 2026-08-09 12:03:47 on dist-test-slave-sb2z"
server_key: "9c73e79483b67df7ccbb8680adb46cd4"
server_key_iv: "64d8a40a59effb411c0dbbf5b1ddec66"
server_key_version: "encryptionkey@0"
I20260809 12:03:47.529785 5714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "f50dd95d153b4cbf83befe29e1cc6808"
format_stamp: "Formatted at 2026-08-09 12:03:47 on dist-test-slave-sb2z"
server_key: "9c73e79483b67df7ccbb8680adb46cd4"
server_key_iv: "64d8a40a59effb411c0dbbf5b1ddec66"
server_key_version: "encryptionkey@0"
I20260809 12:03:47.534960 5714 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 12:03:47.538363 5742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:47.539729 5714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:03:47.539918 5714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "f50dd95d153b4cbf83befe29e1cc6808"
format_stamp: "Formatted at 2026-08-09 12:03:47 on dist-test-slave-sb2z"
server_key: "9c73e79483b67df7ccbb8680adb46cd4"
server_key_iv: "64d8a40a59effb411c0dbbf5b1ddec66"
server_key_version: "encryptionkey@0"
I20260809 12:03:47.540120 5714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:47.563452 5714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:47.567411 5714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:47.567737 5714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:47.585968 5714 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:45273
I20260809 12:03:47.586000 5818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:45273 every 8 connection(s)
I20260809 12:03:47.587296 5714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:47.590891 5819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:47.592905 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 5714
I20260809 12:03:47.593151 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:47.593791 589 external_mini_cluster.cc:1428] Setting key b659cdbea99c57dde691acaa879e46fe
I20260809 12:03:47.604292 5819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808: Bootstrap starting.
I20260809 12:03:47.607815 5819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:47.608810 5819 log.cc:824] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:03:47 dist-test-slave-sb2z krb5kdc[5697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:47.612337 5819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808: No bootstrap required, opened a new log
I20260809 12:03:47.619234 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:47.595884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.0.147.126:45273)
0809 12:03:47.596762 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:47.596777 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:47.596831 (+ 54us) server_negotiation.cc:407] Connection header received
0809 12:03:47.599195 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:47.599222 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:47.599775 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:47.600214 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:47.601530 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:47.603559 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:47.604838 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:47.605906 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:47.606160 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:47.610491 (+ 4331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:47.610502 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:47.610519 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:47.610566 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:47.614753 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:47.615413 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:47.615421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:47.615429 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:47.615709 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:47.616058 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:47.616063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:47.616067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:47.616778 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:47.617108 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:47.617394 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 12:03:47.617848 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":359,"threads_started":1}
I20260809 12:03:47.622999 5819 raft_consensus.cc:348] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50dd95d153b4cbf83befe29e1cc6808" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45273 } }
I20260809 12:03:47.623343 5819 raft_consensus.cc:374] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:47.623420 5819 raft_consensus.cc:729] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50dd95d153b4cbf83befe29e1cc6808, State: Initialized, Role: FOLLOWER
I20260809 12:03:47.624070 5819 consensus_queue.cc:260] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50dd95d153b4cbf83befe29e1cc6808" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45273 } }
I20260809 12:03:47.624369 5819 raft_consensus.cc:388] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:47.624542 5819 raft_consensus.cc:482] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:47.624714 5819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:47.626863 5819 raft_consensus.cc:504] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50dd95d153b4cbf83befe29e1cc6808" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45273 } }
I20260809 12:03:47.627353 5819 leader_election.cc:302] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f50dd95d153b4cbf83befe29e1cc6808; no voters:
I20260809 12:03:47.628068 5819 leader_election.cc:288] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:47.628206 5826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:47.629000 5826 raft_consensus.cc:686] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [term 1 LEADER]: Becoming Leader. State: Replica: f50dd95d153b4cbf83befe29e1cc6808, State: Running, Role: LEADER
I20260809 12:03:47.629629 5826 consensus_queue.cc:237] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50dd95d153b4cbf83befe29e1cc6808" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45273 } }
I20260809 12:03:47.630501 5819 sys_catalog.cc:564] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:47.632400 5827 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f50dd95d153b4cbf83befe29e1cc6808. Latest consensus state: current_term: 1 leader_uuid: "f50dd95d153b4cbf83befe29e1cc6808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50dd95d153b4cbf83befe29e1cc6808" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45273 } } }
I20260809 12:03:47.632411 5828 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f50dd95d153b4cbf83befe29e1cc6808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50dd95d153b4cbf83befe29e1cc6808" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45273 } } }
I20260809 12:03:47.632856 5827 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:47.632856 5828 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:47.635108 5835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:47.640465 5835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:47.649233 5835 catalog_manager.cc:1347] Generated new cluster ID: 60715eba05234eb2a39aa68ea56d56af
I20260809 12:03:47.649343 5835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:47.670131 5835 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:47.671460 5835 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:47.678597 5835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f50dd95d153b4cbf83befe29e1cc6808: Generated new TSK 0
I20260809 12:03:47.679370 5835 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:03:47.745296 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45273
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:38157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:48.089943 5856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:48.090231 5856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:48.090329 5856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:48.097724 5856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:48.097838 5856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:48.097954 5856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:03:48.107605 5856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:38157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:48.108834 5856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:48.110054 5856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:48.125869 5866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:48.125831 5863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:48.126008 5862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:48.126921 5856 server_base.cc:1034] running on GCE node
I20260809 12:03:48.127631 5856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:48.128564 5856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:48.129773 5856 hybrid_clock.cc:648] HybridClock initialized: now 1786277028129721 us; error 38 us; skew 500 ppm
I20260809 12:03:48.130105 5856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:48 dist-test-slave-sb2z krb5kdc[5697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277028, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:48.133502 5856 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:03:48.134974 5856 webserver.cc:460] Webserver started at http://127.0.147.65:35441/ using document root <none> and password file <none>
I20260809 12:03:48.135756 5856 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:48.135917 5856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:48.136283 5856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:48.139351 5856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "fdbd03d8e98e450197355e0370e9a919"
format_stamp: "Formatted at 2026-08-09 12:03:48 on dist-test-slave-sb2z"
server_key: "d8acb24a3751f821e0ffefb8ddeea3bb"
server_key_iv: "577160133e88602075c5d626b65e0518"
server_key_version: "encryptionkey@0"
I20260809 12:03:48.140264 5856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "fdbd03d8e98e450197355e0370e9a919"
format_stamp: "Formatted at 2026-08-09 12:03:48 on dist-test-slave-sb2z"
server_key: "d8acb24a3751f821e0ffefb8ddeea3bb"
server_key_iv: "577160133e88602075c5d626b65e0518"
server_key_version: "encryptionkey@0"
I20260809 12:03:48.145714 5856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:03:48.149247 5875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:48.150550 5856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:03:48.150740 5856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "fdbd03d8e98e450197355e0370e9a919"
format_stamp: "Formatted at 2026-08-09 12:03:48 on dist-test-slave-sb2z"
server_key: "d8acb24a3751f821e0ffefb8ddeea3bb"
server_key_iv: "577160133e88602075c5d626b65e0518"
server_key_version: "encryptionkey@0"
I20260809 12:03:48.150954 5856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:48.178013 5856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:48.182847 5856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:48.183167 5856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:48.184218 5856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:48.185539 5856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:48.185650 5856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:48.185808 5856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:48.185881 5856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:48.220266 5856 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:38857
I20260809 12:03:48.220311 6031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:38857 every 8 connection(s)
I20260809 12:03:48.221602 5856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:03:48.229110 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 5856
I20260809 12:03:48.229348 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:03:48.230019 589 external_mini_cluster.cc:1428] Setting key f28698601d7bd20bcad5c592f7c48991
Aug 09 12:03:48 dist-test-slave-sb2z krb5kdc[5697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277028, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:48.243757 6034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:48.224172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:38273 (local address 127.0.147.126:45273)
0809 12:03:48.224664 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:48.224671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:48.225624 (+ 953us) server_negotiation.cc:407] Connection header received
0809 12:03:48.227412 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:48.227417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:48.227486 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:48.227633 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:48.229352 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:48.230180 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:48.232222 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:48.232810 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:48.232979 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:48.236508 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:48.236517 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:48.236525 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:48.236577 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:48.240218 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:48.240971 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:48.240979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:48.240985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:48.241132 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:48.241741 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:48.241748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:48.241753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:48.242148 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:48.242302 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:48.242959 (+ 657us) server_negotiation.cc:299] Negotiation successful
0809 12:03:48.243217 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":236,"threads_started":1}
I20260809 12:03:48.244424 6035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:48.224755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45273 (local address 127.0.147.65:38273)
0809 12:03:48.225461 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:48.225499 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:48.227165 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:48.227843 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:48.227851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:48.228361 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:48.229158 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:48.229186 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:48.230339 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:48.230344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:48.232075 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:48.232092 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:48.233028 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:48.233034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:48.233240 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:48.233935 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:48.233957 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:48.236238 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:48.240398 (+ 4160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:48.240404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:48.240415 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:48.240851 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:48.241277 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:48.241284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:48.241289 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:48.241582 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:48.242291 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:48.242297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:48.242581 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 12:03:48.242813 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 12:03:48.243193 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":289,"threads_started":1}
I20260809 12:03:48.246055 6032 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45273
I20260809 12:03:48.246368 6032 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:48.247056 6032 heartbeater.cc:511] Master 127.0.147.126:45273 requested a full tablet report, sending...
I20260809 12:03:48.249424 5771 ts_manager.cc:194] Registered new tserver with Master: fdbd03d8e98e450197355e0370e9a919 (127.0.147.65:38857)
I20260809 12:03:48.251672 5771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:38273
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:03:48.282675 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45273
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:38157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:48.609279 6038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:48.609536 6038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:48.609638 6038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:48.616658 6038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:48.616761 6038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:48.616873 6038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:03:48.626096 6038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:38157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:48.627238 6038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:48.628453 6038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:48.641870 6044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:48.642112 6046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:48.642119 6043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:48.643028 6038 server_base.cc:1034] running on GCE node
I20260809 12:03:48.643699 6038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:48.644681 6038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:48.645907 6038 hybrid_clock.cc:648] HybridClock initialized: now 1786277028645861 us; error 38 us; skew 500 ppm
I20260809 12:03:48.646224 6038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:48 dist-test-slave-sb2z krb5kdc[5697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277028, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:48.653527 6038 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:03:48.655114 6038 webserver.cc:460] Webserver started at http://127.0.147.66:43229/ using document root <none> and password file <none>
I20260809 12:03:48.655936 6038 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:48.656078 6038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:48.656391 6038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:48.659212 6038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "441a4eaea31d4d69b977cc8e028dd710"
format_stamp: "Formatted at 2026-08-09 12:03:48 on dist-test-slave-sb2z"
server_key: "34462228c3802c07de6ffa37786103f7"
server_key_iv: "32c9c2adac46174e30659d669b1ffaed"
server_key_version: "encryptionkey@0"
I20260809 12:03:48.660061 6038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "441a4eaea31d4d69b977cc8e028dd710"
format_stamp: "Formatted at 2026-08-09 12:03:48 on dist-test-slave-sb2z"
server_key: "34462228c3802c07de6ffa37786103f7"
server_key_iv: "32c9c2adac46174e30659d669b1ffaed"
server_key_version: "encryptionkey@0"
I20260809 12:03:48.665014 6038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:03:48.668449 6058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:48.669648 6038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:03:48.669811 6038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "441a4eaea31d4d69b977cc8e028dd710"
format_stamp: "Formatted at 2026-08-09 12:03:48 on dist-test-slave-sb2z"
server_key: "34462228c3802c07de6ffa37786103f7"
server_key_iv: "32c9c2adac46174e30659d669b1ffaed"
server_key_version: "encryptionkey@0"
I20260809 12:03:48.670004 6038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:48.697506 6038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:48.701745 6038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:48.702144 6038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:48.703078 6038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:48.704381 6038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:48.704453 6038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:48.704535 6038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:48.704578 6038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:48.738852 6038 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:44029
I20260809 12:03:48.738901 6212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:44029 every 8 connection(s)
I20260809 12:03:48.740216 6038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:03:48.743659 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 6038
I20260809 12:03:48.743918 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:03:48.744544 589 external_mini_cluster.cc:1428] Setting key 1e6c0802e9aa062df445d01d524b29dd
Aug 09 12:03:48 dist-test-slave-sb2z krb5kdc[5697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277028, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:48.762578 6034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:48.742752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:55467 (local address 127.0.147.126:45273)
0809 12:03:48.742972 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:48.742978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:48.744472 (+ 1494us) server_negotiation.cc:407] Connection header received
0809 12:03:48.746490 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:48.746495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:48.746578 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:48.746677 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:48.748866 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:48.749555 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:48.752038 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:48.752479 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:48.752573 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:48.756898 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:48.756903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:48.756906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:48.756938 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:48.759571 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:48.760304 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:48.760310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:48.760313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:48.760450 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:48.760882 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:48.760893 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:48.760896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:48.761162 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:48.761327 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:48.762001 (+ 674us) server_negotiation.cc:299] Negotiation successful
0809 12:03:48.762235 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:03:48.763620 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:48.743354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45273 (local address 127.0.147.66:55467)
0809 12:03:48.744284 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:48.744335 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:48.746238 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:48.746953 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:48.746965 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:48.747808 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:48.748683 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:48.748712 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:48.749715 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:48.749723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:48.751889 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:48.751907 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:48.752712 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:48.752722 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:48.753054 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:48.753829 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:48.753864 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:48.756666 (+ 2802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:48.759720 (+ 3054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:48.759729 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:48.759744 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:48.760172 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:48.760566 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:48.760571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:48.760575 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:48.760758 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:48.761302 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:48.761310 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:48.761639 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 12:03:48.761893 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 12:03:48.762414 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":427,"threads_started":1}
I20260809 12:03:48.765138 6213 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45273
I20260809 12:03:48.765486 6213 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:48.766422 6213 heartbeater.cc:511] Master 127.0.147.126:45273 requested a full tablet report, sending...
I20260809 12:03:48.768260 5771 ts_manager.cc:194] Registered new tserver with Master: 441a4eaea31d4d69b977cc8e028dd710 (127.0.147.66:44029)
I20260809 12:03:48.769506 5771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:55467
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:03:48.800705 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45273
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:38157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:49.138098 6218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:49.138387 6218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:49.138509 6218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:49.146085 6218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:49.146198 6218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:49.146322 6218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:03:49.156240 6218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:38157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:49.157403 6218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:49.158632 6218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:49.171931 6225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:49.171849 6226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:49.171967 6231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:49.172744 6218 server_base.cc:1034] running on GCE node
I20260809 12:03:49.173439 6218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:49.174353 6218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:49.175611 6218 hybrid_clock.cc:648] HybridClock initialized: now 1786277029175547 us; error 47 us; skew 500 ppm
I20260809 12:03:49.175920 6218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[5697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:49.179032 6218 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:03:49.180490 6218 webserver.cc:460] Webserver started at http://127.0.147.67:38467/ using document root <none> and password file <none>
I20260809 12:03:49.181181 6218 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:49.181327 6218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:49.181645 6218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:49.184597 6218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "0e2b332ba4754595b4f8e839177fd8b2"
format_stamp: "Formatted at 2026-08-09 12:03:49 on dist-test-slave-sb2z"
server_key: "5826c88a8efbe8ec4ca61aed09e929b0"
server_key_iv: "f3dd645659ce321328c8adaf6cd06d6a"
server_key_version: "encryptionkey@0"
I20260809 12:03:49.185384 6218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "0e2b332ba4754595b4f8e839177fd8b2"
format_stamp: "Formatted at 2026-08-09 12:03:49 on dist-test-slave-sb2z"
server_key: "5826c88a8efbe8ec4ca61aed09e929b0"
server_key_iv: "f3dd645659ce321328c8adaf6cd06d6a"
server_key_version: "encryptionkey@0"
I20260809 12:03:49.190281 6218 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:03:49.193641 6241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:49.194851 6218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 12:03:49.195031 6218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "0e2b332ba4754595b4f8e839177fd8b2"
format_stamp: "Formatted at 2026-08-09 12:03:49 on dist-test-slave-sb2z"
server_key: "5826c88a8efbe8ec4ca61aed09e929b0"
server_key_iv: "f3dd645659ce321328c8adaf6cd06d6a"
server_key_version: "encryptionkey@0"
I20260809 12:03:49.195252 6218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:49.225356 6218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:49.230077 6218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:49.230405 6218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:49.231422 6218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:49.232818 6218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:49.232919 6218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:49.233074 6218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:49.233156 6218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:49.254751 6032 heartbeater.cc:503] Master 127.0.147.126:45273 was elected leader, sending a full tablet report...
I20260809 12:03:49.270133 6218 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:35501
I20260809 12:03:49.270223 6397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:35501 every 8 connection(s)
I20260809 12:03:49.271412 6218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:03:49.273187 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 6218
I20260809 12:03:49.273388 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:03:49.273870 589 external_mini_cluster.cc:1428] Setting key 720ce2a0a4d1c2c6668c30c723c3039a
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[5697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:49.291579 6401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:49.274079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:56625 (local address 127.0.147.126:45273)
0809 12:03:49.274741 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:49.274746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:49.275957 (+ 1211us) server_negotiation.cc:407] Connection header received
0809 12:03:49.277900 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:49.277903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:49.277966 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:49.278064 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:49.279507 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.280378 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.281895 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.282316 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.282403 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:49.285740 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:49.285745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:49.285748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:49.285780 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:49.288562 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.289240 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.289247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.289250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.289379 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.289832 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.289838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.289841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.290120 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:49.290270 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:49.290928 (+ 658us) server_negotiation.cc:299] Negotiation successful
0809 12:03:49.291154 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":318,"threads_started":1}
I20260809 12:03:49.292483 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:49.274831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45273 (local address 127.0.147.67:56625)
0809 12:03:49.275766 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:49.275836 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:49.277661 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:49.278270 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:49.278278 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:49.278809 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:49.279362 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:49.279378 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.280531 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.280535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:49.281775 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:49.281787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.282495 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.282500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:49.282716 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:49.283294 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:49.283315 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:49.285550 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:49.288702 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:49.288708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:49.288719 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:49.289118 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:49.289492 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:49.289496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:49.289502 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:49.289703 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:49.290247 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:49.290255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:49.290570 (+ 315us) client_negotiation.cc:769] Sending connection context
0809 12:03:49.290830 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 12:03:49.291332 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":445,"threads_started":1}
I20260809 12:03:49.293816 6398 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45273
I20260809 12:03:49.294077 6398 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:49.294698 6398 heartbeater.cc:511] Master 127.0.147.126:45273 requested a full tablet report, sending...
I20260809 12:03:49.296080 5771 ts_manager.cc:194] Registered new tserver with Master: 0e2b332ba4754595b4f8e839177fd8b2 (127.0.147.67:35501)
I20260809 12:03:49.296873 5771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:56625
I20260809 12:03:49.301554 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[5697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[5697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:49.371273 6410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:49.353952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38142 (local address 127.0.147.65:38857)
0809 12:03:49.354705 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:49.354715 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:49.354747 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:03:49.354821 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:49.354832 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:49.355104 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:49.355258 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:49.357617 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.359486 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.361244 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.361830 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.361935 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:49.365940 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:49.365955 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:49.365971 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:49.366043 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:49.368786 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.369382 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.369390 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.369392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.369522 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.369840 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.369843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.369845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.370180 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:49.370405 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:49.370652 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 12:03:49.370849 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":362,"threads_started":1}
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[5697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:49.389233 6401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:49.376164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.0.147.126:45273)
0809 12:03:49.376397 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:49.376403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:49.376500 (+ 97us) server_negotiation.cc:407] Connection header received
0809 12:03:49.376637 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:49.376641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:49.376707 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:49.376787 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:49.377998 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.378707 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.379936 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.380365 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.380461 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:49.384365 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:49.384370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:49.384373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:49.384404 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:49.386929 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.387487 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.387491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.387493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.387678 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.387974 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.387978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.387980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.388298 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:49.388475 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:49.388735 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 12:03:49.388894 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 12:03:49.389860 5771 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53844
I20260809 12:03:49.390817 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 5856
I20260809 12:03:49.403277 6410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:49.391653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38144 (local address 127.0.147.65:38857)
0809 12:03:49.391876 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:49.391882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:49.392116 (+ 234us) server_negotiation.cc:407] Connection header received
0809 12:03:49.392216 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:49.392220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:49.392312 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:49.392390 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:49.393497 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.394156 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.395292 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.395758 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.395852 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:49.398222 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:49.398227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:49.398230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:49.398261 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:49.401010 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.401583 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.401587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.401589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.401707 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.402055 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.402059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.402061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.402305 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:49.402485 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:49.402735 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 12:03:49.402893 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:03:49.403931 6026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:49.486907 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 5856
I20260809 12:03:49.510099 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 6038
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[5697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:03:49.526218 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:49.511063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.0.147.66:44029)
0809 12:03:49.511670 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:49.511678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:49.511705 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:03:49.511766 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:49.511772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:49.511980 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:49.512153 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:49.513454 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.514461 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.515799 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.516341 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.516488 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:49.520633 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:49.520644 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:49.520658 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:49.520690 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:49.523360 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.524052 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.524061 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.524064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.524199 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.524592 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.524596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.524598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.524929 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:49.525262 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:49.525594 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 12:03:49.525870 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":272,"threads_started":1}
I20260809 12:03:49.527581 6206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:49.615685 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 6038
I20260809 12:03:49.640856 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 6218
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[5697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:03:49.657364 6421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:49.641869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32898 (local address 127.0.147.67:35501)
0809 12:03:49.642517 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:49.642525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:49.642555 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:03:49.642613 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:49.642619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:49.642825 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:49.642948 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:49.644532 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.645640 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.646880 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:49.647420 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:49.647593 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:49.652010 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:49.652023 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:49.652037 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:49.652092 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:49.654718 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.655468 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.655475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.655477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.655652 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:49.655978 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:49.655981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:49.655983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:49.656305 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:49.656550 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:49.656773 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 12:03:49.656968 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":294,"threads_started":1}
I20260809 12:03:49.658914 6391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:49.742986 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 6218
I20260809 12:03:49.765798 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 5714
I20260809 12:03:49.767128 5812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:49.873873 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 5714
2026-08-09T12:03:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4242 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[6427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[6427](info): set up 2 sockets
Aug 09 12:03:49 dist-test-slave-sb2z krb5kdc[6427](info): commencing operation
krb5kdc: starting...
W20260809 12:03:51.159612 589 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.007s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:03:51 dist-test-slave-sb2z krb5kdc[6427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:03:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:03:51.401417 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:44441
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:44441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:51.728847 6445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:51.729173 6445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:51.729355 6445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:51.736581 6445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:51.736719 6445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:51.736825 6445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:51.736932 6445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:51.737025 6445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:51.746536 6445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:44441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:44441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:51.747947 6445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:51.749274 6445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:51.761315 6457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:51.761229 6454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:51.761674 6452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:51.762360 6445 server_base.cc:1034] running on GCE node
I20260809 12:03:51.763206 6445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:51.764750 6445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:51.765992 6445 hybrid_clock.cc:648] HybridClock initialized: now 1786277031765950 us; error 55 us; skew 500 ppm
I20260809 12:03:51.766358 6445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:51 dist-test-slave-sb2z krb5kdc[6427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277031, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:51.769646 6445 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:03:51.770790 6445 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:51.770880 6445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:51.771162 6445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:51.774183 6445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "f1e35c3c982f4b46a466c3c49e7ebaea"
format_stamp: "Formatted at 2026-08-09 12:03:51 on dist-test-slave-sb2z"
server_key: "4fd57c2daede71649d54b047526c50cf"
server_key_iv: "6204509cc943b9ef3b8d3f33f818aeed"
server_key_version: "encryptionkey@0"
I20260809 12:03:51.774941 6445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "f1e35c3c982f4b46a466c3c49e7ebaea"
format_stamp: "Formatted at 2026-08-09 12:03:51 on dist-test-slave-sb2z"
server_key: "4fd57c2daede71649d54b047526c50cf"
server_key_iv: "6204509cc943b9ef3b8d3f33f818aeed"
server_key_version: "encryptionkey@0"
I20260809 12:03:51.779938 6445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:03:51.783356 6464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:51.784744 6445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:03:51.784898 6445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "f1e35c3c982f4b46a466c3c49e7ebaea"
format_stamp: "Formatted at 2026-08-09 12:03:51 on dist-test-slave-sb2z"
server_key: "4fd57c2daede71649d54b047526c50cf"
server_key_iv: "6204509cc943b9ef3b8d3f33f818aeed"
server_key_version: "encryptionkey@0"
I20260809 12:03:51.785092 6445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:51.802320 6445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:51.806783 6445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:51.807026 6445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:51.826339 6445 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:44441
I20260809 12:03:51.826382 6537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:44441 every 8 connection(s)
I20260809 12:03:51.827325 6445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:51.831110 6538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:51.831806 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 6445
I20260809 12:03:51.832049 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:51.832748 589 external_mini_cluster.cc:1428] Setting key 65ff560784f45b4eb77e9a6d78467ae5
I20260809 12:03:51.845418 6538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea: Bootstrap starting.
Aug 09 12:03:51 dist-test-slave-sb2z krb5kdc[6427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:51.849404 6538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:51.850747 6538 log.cc:824] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:51.853693 6538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea: No bootstrap required, opened a new log
I20260809 12:03:51.856408 6541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:51.834837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43850 (local address 127.0.147.126:44441)
0809 12:03:51.835775 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:51.835787 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:51.835829 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:03:51.837438 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:51.837461 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:51.837858 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:51.838194 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:51.840160 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:51.841473 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:51.843341 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:51.844040 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:51.844163 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:51.848979 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:51.848987 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:51.849001 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:51.849036 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:51.852304 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:51.852980 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:51.852992 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:51.852998 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:51.853169 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:51.853565 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:51.853572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:51.853577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:51.854171 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:51.854491 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:51.854926 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 12:03:51.855357 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":418,"threads_started":1}
I20260809 12:03:51.862690 6538 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e35c3c982f4b46a466c3c49e7ebaea" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44441 } }
I20260809 12:03:51.863070 6538 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:51.863188 6538 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e35c3c982f4b46a466c3c49e7ebaea, State: Initialized, Role: FOLLOWER
I20260809 12:03:51.863845 6538 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e35c3c982f4b46a466c3c49e7ebaea" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44441 } }
I20260809 12:03:51.864089 6538 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:51.864239 6538 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:51.864423 6538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:51.866603 6538 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e35c3c982f4b46a466c3c49e7ebaea" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44441 } }
I20260809 12:03:51.867136 6538 leader_election.cc:302] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1e35c3c982f4b46a466c3c49e7ebaea; no voters:
I20260809 12:03:51.867765 6538 leader_election.cc:288] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:51.867887 6544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:51.868587 6544 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [term 1 LEADER]: Becoming Leader. State: Replica: f1e35c3c982f4b46a466c3c49e7ebaea, State: Running, Role: LEADER
I20260809 12:03:51.869292 6544 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e35c3c982f4b46a466c3c49e7ebaea" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44441 } }
I20260809 12:03:51.870661 6538 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:51.872500 6546 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1e35c3c982f4b46a466c3c49e7ebaea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e35c3c982f4b46a466c3c49e7ebaea" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44441 } } }
I20260809 12:03:51.872877 6546 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:51.872565 6545 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1e35c3c982f4b46a466c3c49e7ebaea. Latest consensus state: current_term: 1 leader_uuid: "f1e35c3c982f4b46a466c3c49e7ebaea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e35c3c982f4b46a466c3c49e7ebaea" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44441 } } }
I20260809 12:03:51.872999 6545 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:51.874948 6549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:51.881335 6549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:51.890575 6549 catalog_manager.cc:1347] Generated new cluster ID: 985abd5f7b2c42c3a573fb84de61d437
I20260809 12:03:51.890681 6549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:51.913750 6549 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:51.915076 6549 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:51.920029 6549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea: Generated new TSK 0
I20260809 12:03:51.921249 6549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:03:51.979009 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:44441
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:52.315151 6573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:52.315423 6573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:52.315629 6573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:52.322907 6573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:52.323007 6573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:52.323184 6573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:03:52.332811 6573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:44441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:52.334098 6573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:52.335371 6573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:52.348258 6578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:52.348333 6579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:52.348327 6581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:52.349392 6573 server_base.cc:1034] running on GCE node
I20260809 12:03:52.349992 6573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:52.351011 6573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:52.352232 6573 hybrid_clock.cc:648] HybridClock initialized: now 1786277032352184 us; error 42 us; skew 500 ppm
I20260809 12:03:52.352538 6573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:52 dist-test-slave-sb2z krb5kdc[6427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:52.355674 6573 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:03:52.356832 6573 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:52.356966 6573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:52.357290 6573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:52.360330 6573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "0cf567d59a2b434798fe9786884e57d2"
format_stamp: "Formatted at 2026-08-09 12:03:52 on dist-test-slave-sb2z"
server_key: "c87f81d348b01a3d4437a922568b560b"
server_key_iv: "66c7b1cf951fbafa15a298bcd5ee57ea"
server_key_version: "encryptionkey@0"
I20260809 12:03:52.361212 6573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "0cf567d59a2b434798fe9786884e57d2"
format_stamp: "Formatted at 2026-08-09 12:03:52 on dist-test-slave-sb2z"
server_key: "c87f81d348b01a3d4437a922568b560b"
server_key_iv: "66c7b1cf951fbafa15a298bcd5ee57ea"
server_key_version: "encryptionkey@0"
I20260809 12:03:52.366175 6573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:03:52.369498 6591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:52.370769 6573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:03:52.370941 6573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "0cf567d59a2b434798fe9786884e57d2"
format_stamp: "Formatted at 2026-08-09 12:03:52 on dist-test-slave-sb2z"
server_key: "c87f81d348b01a3d4437a922568b560b"
server_key_iv: "66c7b1cf951fbafa15a298bcd5ee57ea"
server_key_version: "encryptionkey@0"
I20260809 12:03:52.371160 6573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:52.383625 6573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:52.387303 6573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:52.387593 6573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:52.388567 6573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:52.389927 6573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:52.390038 6573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:52.390164 6573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:52.390269 6573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:52.425058 6573 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:37553
I20260809 12:03:52.425150 6753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:37553 every 8 connection(s)
I20260809 12:03:52.425945 6573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:03:52.432655 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 6573
I20260809 12:03:52.432875 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:03:52.433523 589 external_mini_cluster.cc:1428] Setting key e255abf9629a30176e1d83087ca17c21
Aug 09 12:03:52 dist-test-slave-sb2z krb5kdc[6427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:52.446672 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:52.428364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:54219 (local address 127.0.147.126:44441)
0809 12:03:52.428809 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:52.428813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:52.429666 (+ 853us) server_negotiation.cc:407] Connection header received
0809 12:03:52.431511 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:52.431517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:52.431640 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:52.431766 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:52.433347 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:52.434131 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:52.436016 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:52.436408 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:52.436542 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:52.440484 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:52.440489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:52.440492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:52.440523 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:52.443174 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:52.444147 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:52.444151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:52.444154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:52.444268 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:52.444714 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:52.444720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:52.444725 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:52.445098 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:52.445261 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:52.445923 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 12:03:52.446174 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":249,"threads_started":1}
I20260809 12:03:52.447214 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:52.428801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:44441 (local address 127.0.147.65:54219)
0809 12:03:52.429502 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:52.429539 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:52.431235 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:52.431996 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:52.432005 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:52.432516 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:52.433174 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:52.433193 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:52.434293 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:52.434303 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:52.435865 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:52.435877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:52.436655 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:52.436662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:52.436867 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:52.437548 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:52.437577 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:52.440278 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:52.443317 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:52.443328 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:52.443346 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:52.444018 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:52.444390 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:52.444398 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:52.444404 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:52.444587 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:52.445227 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:52.445233 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:52.445537 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 12:03:52.445758 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 12:03:52.446124 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":302,"threads_started":1}
I20260809 12:03:52.448863 6755 heartbeater.cc:347] Connected to a master server at 127.0.147.126:44441
I20260809 12:03:52.449167 6755 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:52.449795 6755 heartbeater.cc:511] Master 127.0.147.126:44441 requested a full tablet report, sending...
I20260809 12:03:52.451936 6487 ts_manager.cc:194] Registered new tserver with Master: 0cf567d59a2b434798fe9786884e57d2 (127.0.147.65:37553)
I20260809 12:03:52.453361 6487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:54219
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:03:52.488543 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:44441
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:52.821931 6762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:52.822219 6762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:52.822420 6762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:52.832124 6762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:52.832265 6762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:52.832468 6762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:03:52.846133 6762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:44441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:52.847396 6762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:52.848719 6762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:52.861374 6772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:52.861464 6775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:52.861481 6773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:52.862354 6762 server_base.cc:1034] running on GCE node
I20260809 12:03:52.863036 6762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:52.864061 6762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:52.865300 6762 hybrid_clock.cc:648] HybridClock initialized: now 1786277032865256 us; error 55 us; skew 500 ppm
I20260809 12:03:52.865624 6762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:52 dist-test-slave-sb2z krb5kdc[6427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:52.868837 6762 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:03:52.870008 6762 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:52.870141 6762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:52.870505 6762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:52.873970 6762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "92d3a6d58ae84f908ee5134bc42c3265"
format_stamp: "Formatted at 2026-08-09 12:03:52 on dist-test-slave-sb2z"
server_key: "1b75e9549973cdcfaecdd9b5e31d2a3d"
server_key_iv: "4a06d15458c09bb56cda56bf59e66b98"
server_key_version: "encryptionkey@0"
I20260809 12:03:52.875088 6762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "92d3a6d58ae84f908ee5134bc42c3265"
format_stamp: "Formatted at 2026-08-09 12:03:52 on dist-test-slave-sb2z"
server_key: "1b75e9549973cdcfaecdd9b5e31d2a3d"
server_key_iv: "4a06d15458c09bb56cda56bf59e66b98"
server_key_version: "encryptionkey@0"
I20260809 12:03:52.880373 6762 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:03:52.883816 6783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:52.885195 6762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:03:52.885399 6762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "92d3a6d58ae84f908ee5134bc42c3265"
format_stamp: "Formatted at 2026-08-09 12:03:52 on dist-test-slave-sb2z"
server_key: "1b75e9549973cdcfaecdd9b5e31d2a3d"
server_key_iv: "4a06d15458c09bb56cda56bf59e66b98"
server_key_version: "encryptionkey@0"
I20260809 12:03:52.885636 6762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:52.900669 6762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:52.904654 6762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:52.904948 6762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:52.905851 6762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:52.907148 6762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:52.907263 6762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:52.907406 6762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:52.907516 6762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:52.942943 6762 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:36503
I20260809 12:03:52.943001 6931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:36503 every 8 connection(s)
I20260809 12:03:52.943910 6762 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:03:52.952154 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 6762
I20260809 12:03:52.952371 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:03:52.952844 589 external_mini_cluster.cc:1428] Setting key 315fc37eb359e7e584e7f39fc9370017
Aug 09 12:03:52 dist-test-slave-sb2z krb5kdc[6427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:52.964896 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:52.946328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:59737 (local address 127.0.147.126:44441)
0809 12:03:52.946508 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:52.946512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:52.947634 (+ 1122us) server_negotiation.cc:407] Connection header received
0809 12:03:52.949475 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:52.949481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:52.949581 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:52.949698 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:52.951171 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:52.951979 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:52.953827 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:52.954300 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:52.954439 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:52.958035 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:52.958045 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:52.958053 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:52.958099 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:52.961725 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:52.962432 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:52.962440 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:52.962446 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:52.962613 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:52.963006 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:52.963020 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:52.963027 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:52.963410 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:52.963593 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:52.964263 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 12:03:52.964524 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:03:52.965973 6938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:52.946719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:44441 (local address 127.0.147.66:59737)
0809 12:03:52.947414 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:52.947462 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:52.949202 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:52.949926 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:52.949934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:52.950442 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:52.950984 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:52.951000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:52.952148 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:52.952152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:52.953678 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:52.953691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:52.954532 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:52.954539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:52.954773 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:52.955466 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:52.955494 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:52.957751 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:52.961869 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:52.961876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:52.961887 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:52.962313 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:52.962726 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:52.962730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:52.962732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:52.962900 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:52.963577 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:52.963589 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:52.963892 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 12:03:52.964140 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 12:03:52.964516 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":286,"threads_started":1}
I20260809 12:03:52.967577 6935 heartbeater.cc:347] Connected to a master server at 127.0.147.126:44441
I20260809 12:03:52.967896 6935 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:52.968595 6935 heartbeater.cc:511] Master 127.0.147.126:44441 requested a full tablet report, sending...
I20260809 12:03:52.970091 6487 ts_manager.cc:194] Registered new tserver with Master: 92d3a6d58ae84f908ee5134bc42c3265 (127.0.147.66:36503)
I20260809 12:03:52.971361 6487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:59737
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:03:53.008868 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:44441
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:03:53.341566 6942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:53.341830 6942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:53.341990 6942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:53.349411 6942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:53.349519 6942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:53.349730 6942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:03:53.359331 6942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:44441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:53.360610 6942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:53.361917 6942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:53.374653 6949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:53.374872 6952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:53.375218 6950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:53.375643 6942 server_base.cc:1034] running on GCE node
I20260809 12:03:53.376312 6942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:53.377307 6942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:53.378573 6942 hybrid_clock.cc:648] HybridClock initialized: now 1786277033378514 us; error 46 us; skew 500 ppm
I20260809 12:03:53.378922 6942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:03:53 dist-test-slave-sb2z krb5kdc[6427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277033, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:03:53.382125 6942 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:03:53.383440 6942 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:53.383605 6942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:53.383926 6942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:53.386736 6942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "0f79fe5013264cd28bec6eb7718c8e94"
format_stamp: "Formatted at 2026-08-09 12:03:53 on dist-test-slave-sb2z"
server_key: "4e7614d3e20f9a3a1bb7f1ee8d37f562"
server_key_iv: "495e6e02d8413989fb25fbddce929041"
server_key_version: "encryptionkey@0"
I20260809 12:03:53.387604 6942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "0f79fe5013264cd28bec6eb7718c8e94"
format_stamp: "Formatted at 2026-08-09 12:03:53 on dist-test-slave-sb2z"
server_key: "4e7614d3e20f9a3a1bb7f1ee8d37f562"
server_key_iv: "495e6e02d8413989fb25fbddce929041"
server_key_version: "encryptionkey@0"
I20260809 12:03:53.392709 6942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:03:53.396118 6961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:53.397343 6942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:03:53.397521 6942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "0f79fe5013264cd28bec6eb7718c8e94"
format_stamp: "Formatted at 2026-08-09 12:03:53 on dist-test-slave-sb2z"
server_key: "4e7614d3e20f9a3a1bb7f1ee8d37f562"
server_key_iv: "495e6e02d8413989fb25fbddce929041"
server_key_version: "encryptionkey@0"
I20260809 12:03:53.397743 6942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:53.416884 6942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:53.420789 6942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:53.421096 6942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:53.422042 6942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:53.423388 6942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:53.423518 6942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:53.423707 6942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:53.423802 6942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:53.456441 6755 heartbeater.cc:503] Master 127.0.147.126:44441 was elected leader, sending a full tablet report...
I20260809 12:03:53.460364 6942 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:42013
I20260809 12:03:53.460410 7111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:42013 every 8 connection(s)
I20260809 12:03:53.461239 6942 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:03:53.471546 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 6942
I20260809 12:03:53.471771 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:03:53.472334 589 external_mini_cluster.cc:1428] Setting key 645c3ef9c825b010319ddbc4a71ddf48
Aug 09 12:03:53 dist-test-slave-sb2z krb5kdc[6427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277033, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:03:53.481128 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.463659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:45339 (local address 127.0.147.126:44441)
0809 12:03:53.463825 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:53.463829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:53.464968 (+ 1139us) server_negotiation.cc:407] Connection header received
0809 12:03:53.466884 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:53.466889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:53.466980 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:53.467099 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:53.468656 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.469407 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.470949 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.471660 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.471800 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.475701 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:53.475705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:53.475709 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:53.475742 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:53.478475 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:53.479163 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:53.479167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:53.479169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:53.479291 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:53.479719 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:53.479722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:53.479724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:53.479976 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:53.480095 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:53.480701 (+ 606us) server_negotiation.cc:299] Negotiation successful
0809 12:03:53.480855 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:03:53.482075 7115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.464121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:44441 (local address 127.0.147.67:45339)
0809 12:03:53.464809 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:53.464853 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:53.466613 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:53.467325 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:53.467333 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:53.467921 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:53.468466 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.468483 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.469577 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.469581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.470802 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.470814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.471924 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.471932 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.472143 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.472902 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:03:53.472924 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:53.475464 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:53.478600 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:53.478606 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:53.478617 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:53.479053 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:53.479381 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:03:53.479385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:03:53.479387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:03:53.479629 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:03:53.480096 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:53.480101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:53.480376 (+ 275us) client_negotiation.cc:769] Sending connection context
0809 12:03:53.480591 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 12:03:53.480967 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":274,"threads_started":1}
I20260809 12:03:53.483465 7112 heartbeater.cc:347] Connected to a master server at 127.0.147.126:44441
I20260809 12:03:53.483839 7112 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:53.484511 7112 heartbeater.cc:511] Master 127.0.147.126:44441 requested a full tablet report, sending...
I20260809 12:03:53.485911 6487 ts_manager.cc:194] Registered new tserver with Master: 0f79fe5013264cd28bec6eb7718c8e94 (127.0.147.67:42013)
I20260809 12:03:53.486975 6487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:45339
I20260809 12:03:53.487771 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:03:53.505357 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.492987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43852 (local address 127.0.147.126:44441)
0809 12:03:53.493175 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:53.493184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:53.493694 (+ 510us) server_negotiation.cc:407] Connection header received
0809 12:03:53.493802 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:53.493806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:53.493862 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:53.493940 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:53.495404 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.496170 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.497372 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.497780 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.497872 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.500197 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:53.500202 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:53.500205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:53.500233 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:53.502872 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:53.503467 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:53.503471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:53.503473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:53.503669 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:53.503993 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:53.503996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:53.503998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:53.504278 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:53.504497 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:53.504834 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 12:03:53.504986 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:03:53.509495 6485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:03:53.516721 6485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:53.539371 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.529240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.0.147.65:37553)
0809 12:03:53.530568 (+ 1328us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:53.530578 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:53.530611 (+ 33us) server_negotiation.cc:407] Connection header received
0809 12:03:53.530724 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:53.530732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:53.531024 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:53.532110 (+ 1086us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:53.532363 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.533970 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.536767 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.538138 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.538289 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.538532 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:53.538861 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 12:03:53.539047 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1141,"thread_start_us":995,"threads_started":1}
I20260809 12:03:53.539397 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.528931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:37553 (local address 127.0.0.1:46954)
0809 12:03:53.529946 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:53.529983 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:53.530202 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:53.531488 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:53.531496 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:53.531562 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:53.531905 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.531914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.534112 (+ 2198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.534118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.536541 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.536568 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.538359 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.538366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.538571 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.538629 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 12:03:53.538780 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 12:03:53.538969 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":610,"threads_started":1}
I20260809 12:03:53.540271 7135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.530806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58640 (local address 127.0.147.66:36503)
0809 12:03:53.531746 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:53.531755 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:53.531800 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:03:53.531914 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:53.531923 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:53.532213 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:53.533338 (+ 1125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:53.533603 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.535304 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.537778 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.539028 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.539439 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.539658 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:53.539807 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 12:03:53.539920 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":597,"threads_started":1}
I20260809 12:03:53.540371 7134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.530311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36503 (local address 127.0.0.1:58640)
0809 12:03:53.531028 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:53.531046 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:53.531167 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:53.532546 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:53.532554 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:53.532585 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:53.533100 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.533116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.535553 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.535560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.537620 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.537632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.539281 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.539288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.539574 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.539624 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 12:03:53.539726 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:03:53.539858 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":26,"thread_start_us":442,"threads_started":1}
I20260809 12:03:53.543828 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.531116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:42013 (local address 127.0.0.1:40362)
0809 12:03:53.534070 (+ 2954us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:53.534090 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:53.534213 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:53.534742 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:53.534746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:53.534770 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:53.535212 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.535222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.537266 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.537272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.541173 (+ 3901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.541194 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.543011 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.543018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.543181 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.543216 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:03:53.543314 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 12:03:53.543435 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2807,"mutex_wait_us":5,"thread_start_us":2686,"threads_started":1}
I20260809 12:03:53.543915 7138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.531716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40362 (local address 127.0.147.67:42013)
0809 12:03:53.533643 (+ 1927us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:53.533653 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:53.534317 (+ 664us) server_negotiation.cc:407] Connection header received
0809 12:03:53.534395 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:53.534400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:53.534559 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:53.534747 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:53.535394 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.537112 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.541336 (+ 4224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.542793 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.542924 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.543136 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:53.543374 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 12:03:53.543554 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1765,"thread_start_us":1598,"threads_started":1}
I20260809 12:03:53.543883 6658 tablet_service.cc:1467] Processing CreateTablet for tablet 4049bcbc32544e168b81ae8e3e2f3113 (DEFAULT_TABLE table=test-table [id=48dadab510bc480aad4a6d23b4ae3b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:53.544281 6846 tablet_service.cc:1467] Processing CreateTablet for tablet 4049bcbc32544e168b81ae8e3e2f3113 (DEFAULT_TABLE table=test-table [id=48dadab510bc480aad4a6d23b4ae3b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:53.545863 6846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4049bcbc32544e168b81ae8e3e2f3113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:53.545864 6658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4049bcbc32544e168b81ae8e3e2f3113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:53.547559 7030 tablet_service.cc:1467] Processing CreateTablet for tablet 4049bcbc32544e168b81ae8e3e2f3113 (DEFAULT_TABLE table=test-table [id=48dadab510bc480aad4a6d23b4ae3b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:53.548897 7030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4049bcbc32544e168b81ae8e3e2f3113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:53.557487 7140 tablet_bootstrap.cc:492] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: Bootstrap starting.
I20260809 12:03:53.560954 7142 tablet_bootstrap.cc:492] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: Bootstrap starting.
I20260809 12:03:53.562536 7140 tablet_bootstrap.cc:654] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:53.563500 7143 tablet_bootstrap.cc:492] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: Bootstrap starting.
I20260809 12:03:53.563891 7140 log.cc:824] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:53.564848 7142 tablet_bootstrap.cc:654] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:53.566097 7142 log.cc:824] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:53.567058 7143 tablet_bootstrap.cc:654] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:53.567689 7140 tablet_bootstrap.cc:492] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: No bootstrap required, opened a new log
I20260809 12:03:53.568025 7140 ts_tablet_manager.cc:1397] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260809 12:03:53.568478 7143 log.cc:824] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:53.569998 7142 tablet_bootstrap.cc:492] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: No bootstrap required, opened a new log
I20260809 12:03:53.570297 7142 ts_tablet_manager.cc:1397] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 12:03:53.571275 7143 tablet_bootstrap.cc:492] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: No bootstrap required, opened a new log
I20260809 12:03:53.571647 7143 ts_tablet_manager.cc:1397] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260809 12:03:53.578842 7140 raft_consensus.cc:348] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } }
I20260809 12:03:53.579276 7140 raft_consensus.cc:374] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:53.579424 7140 raft_consensus.cc:729] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d3a6d58ae84f908ee5134bc42c3265, State: Initialized, Role: FOLLOWER
I20260809 12:03:53.579650 7143 raft_consensus.cc:348] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } }
I20260809 12:03:53.580091 7143 raft_consensus.cc:374] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:53.580212 7143 raft_consensus.cc:729] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f79fe5013264cd28bec6eb7718c8e94, State: Initialized, Role: FOLLOWER
I20260809 12:03:53.580180 7140 consensus_queue.cc:260] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } }
I20260809 12:03:53.580983 7143 consensus_queue.cc:260] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } }
I20260809 12:03:53.581362 6935 heartbeater.cc:503] Master 127.0.147.126:44441 was elected leader, sending a full tablet report...
I20260809 12:03:53.581727 7142 raft_consensus.cc:348] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } }
I20260809 12:03:53.582075 7140 ts_tablet_manager.cc:1428] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260809 12:03:53.582268 7142 raft_consensus.cc:374] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:53.582441 7142 raft_consensus.cc:729] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf567d59a2b434798fe9786884e57d2, State: Initialized, Role: FOLLOWER
I20260809 12:03:53.583082 7112 heartbeater.cc:503] Master 127.0.147.126:44441 was elected leader, sending a full tablet report...
I20260809 12:03:53.583243 7142 consensus_queue.cc:260] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } }
I20260809 12:03:53.583770 7143 ts_tablet_manager.cc:1428] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260809 12:03:53.585438 7142 ts_tablet_manager.cc:1428] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: Time spent starting tablet: real 0.015s user 0.017s sys 0.000s
W20260809 12:03:53.678159 6756 tablet.cc:2369] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:03:53.695858 6937 tablet.cc:2369] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:03:53.712827 7114 tablet.cc:2369] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:03:53.733738 7152 raft_consensus.cc:482] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:53.734078 7152 raft_consensus.cc:504] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } }
I20260809 12:03:53.735828 7152 leader_election.cc:288] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d3a6d58ae84f908ee5134bc42c3265 (127.0.147.66:36503), 0f79fe5013264cd28bec6eb7718c8e94 (127.0.147.67:42013)
I20260809 12:03:53.743240 7135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.736262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:47137 (local address 127.0.147.66:36503)
0809 12:03:53.736487 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:53.736495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:53.737169 (+ 674us) server_negotiation.cc:407] Connection header received
0809 12:03:53.737424 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:53.737429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:53.737513 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:53.737585 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:53.738468 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.739312 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.741328 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.742198 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.742323 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.742376 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:53.742823 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 12:03:53.742958 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:03:53.743242 7138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.737396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:36111 (local address 127.0.147.67:42013)
0809 12:03:53.737609 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:53.737615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:53.737853 (+ 238us) server_negotiation.cc:407] Connection header received
0809 12:03:53.737994 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:53.738001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:53.738074 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:53.738181 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:03:53.738801 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.739651 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.741510 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.742336 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.742509 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.742586 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:53.742877 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 12:03:53.743009 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:03:53.743342 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.736273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36503 (local address 127.0.147.65:47137)
0809 12:03:53.737012 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:53.737051 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:53.737222 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:53.737788 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:53.737792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:53.737840 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:53.738280 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.738297 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.739459 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.739465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.741147 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.741159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.742431 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.742440 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.742649 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.742673 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:03:53.742747 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 12:03:53.742875 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":25,"thread_start_us":419,"threads_started":1}
I20260809 12:03:53.743413 7158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.737108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:42013 (local address 127.0.147.65:36111)
0809 12:03:53.737744 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:53.737759 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:53.737871 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:53.738202 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:53.738205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:53.738223 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:03:53.738638 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.738650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.739793 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.739797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.741350 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:53.741365 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.742575 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.742579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:53.742692 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.742714 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 12:03:53.742835 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 12:03:53.742956 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":37,"thread_start_us":307,"threads_started":1}
I20260809 12:03:53.744765 7050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4049bcbc32544e168b81ae8e3e2f3113" candidate_uuid: "0cf567d59a2b434798fe9786884e57d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f79fe5013264cd28bec6eb7718c8e94" is_pre_election: true
I20260809 12:03:53.744756 6873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4049bcbc32544e168b81ae8e3e2f3113" candidate_uuid: "0cf567d59a2b434798fe9786884e57d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d3a6d58ae84f908ee5134bc42c3265" is_pre_election: true
I20260809 12:03:53.745388 7050 raft_consensus.cc:2454] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cf567d59a2b434798fe9786884e57d2 in term 0.
I20260809 12:03:53.745388 6873 raft_consensus.cc:2454] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cf567d59a2b434798fe9786884e57d2 in term 0.
I20260809 12:03:53.746111 6595 leader_election.cc:302] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf567d59a2b434798fe9786884e57d2, 0f79fe5013264cd28bec6eb7718c8e94; no voters:
I20260809 12:03:53.746498 7152 raft_consensus.cc:2781] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:53.746667 7152 raft_consensus.cc:482] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:53.746790 7152 raft_consensus.cc:3037] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:53.749059 7152 raft_consensus.cc:504] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } }
I20260809 12:03:53.749662 7152 leader_election.cc:288] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [CANDIDATE]: Term 1 election: Requested vote from peers 92d3a6d58ae84f908ee5134bc42c3265 (127.0.147.66:36503), 0f79fe5013264cd28bec6eb7718c8e94 (127.0.147.67:42013)
I20260809 12:03:53.750195 6873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4049bcbc32544e168b81ae8e3e2f3113" candidate_uuid: "0cf567d59a2b434798fe9786884e57d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d3a6d58ae84f908ee5134bc42c3265"
I20260809 12:03:53.750227 7050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4049bcbc32544e168b81ae8e3e2f3113" candidate_uuid: "0cf567d59a2b434798fe9786884e57d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f79fe5013264cd28bec6eb7718c8e94"
I20260809 12:03:53.750415 7050 raft_consensus.cc:3037] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:53.750432 6873 raft_consensus.cc:3037] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:53.753844 6873 raft_consensus.cc:2454] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cf567d59a2b434798fe9786884e57d2 in term 1.
I20260809 12:03:53.753849 7050 raft_consensus.cc:2454] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cf567d59a2b434798fe9786884e57d2 in term 1.
I20260809 12:03:53.754297 6595 leader_election.cc:302] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf567d59a2b434798fe9786884e57d2, 92d3a6d58ae84f908ee5134bc42c3265; no voters:
I20260809 12:03:53.754580 7152 raft_consensus.cc:2781] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:53.755045 7152 raft_consensus.cc:686] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 1 LEADER]: Becoming Leader. State: Replica: 0cf567d59a2b434798fe9786884e57d2, State: Running, Role: LEADER
I20260809 12:03:53.755609 7152 consensus_queue.cc:237] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } }
I20260809 12:03:53.760274 6485 catalog_manager.cc:5495] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cf567d59a2b434798fe9786884e57d2 (127.0.147.65). New cstate: current_term: 1 leader_uuid: "0cf567d59a2b434798fe9786884e57d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf567d59a2b434798fe9786884e57d2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 37553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 } health_report { overall_health: UNKNOWN } } }
I20260809 12:03:53.832789 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.827307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46958 (local address 127.0.147.65:37553)
0809 12:03:53.827578 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:53.827585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:53.827607 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:03:53.827811 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:53.827815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:53.827910 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:53.828072 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:03:53.828483 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.829209 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.830592 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.831015 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.831148 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.831686 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:03:53.832197 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:03:53.832359 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:53.832441 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 12:03:53.832534 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 12:03:53.842614 7050 raft_consensus.cc:1261] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [term 1 FOLLOWER]: Refusing update from remote peer 0cf567d59a2b434798fe9786884e57d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:53.842614 6873 raft_consensus.cc:1261] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [term 1 FOLLOWER]: Refusing update from remote peer 0cf567d59a2b434798fe9786884e57d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:53.843684 7159 consensus_queue.cc:1035] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d3a6d58ae84f908ee5134bc42c3265" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:53.844074 7152 consensus_queue.cc:1035] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f79fe5013264cd28bec6eb7718c8e94" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 42013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:53.860890 7173 mvcc.cc:205] Tried to move back new op lower bound from 7316590730607632384 to 7316590730266980352. Current Snapshot: MvccSnapshot[applied={T|T < 7316590730607632384}]
I20260809 12:03:53.861390 7174 mvcc.cc:205] Tried to move back new op lower bound from 7316590730607632384 to 7316590730266980352. Current Snapshot: MvccSnapshot[applied={T|T < 7316590730607632384}]
I20260809 12:03:53.870019 6485 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:53.870276 6485 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:53.873831 6485 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1e35c3c982f4b46a466c3c49e7ebaea: Sending DeleteTablet for 3 replicas of tablet 4049bcbc32544e168b81ae8e3e2f3113
I20260809 12:03:53.874971 6658 tablet_service.cc:1514] Processing DeleteTablet for tablet 4049bcbc32544e168b81ae8e3e2f3113 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:53 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:46954
I20260809 12:03:53.875124 7030 tablet_service.cc:1514] Processing DeleteTablet for tablet 4049bcbc32544e168b81ae8e3e2f3113 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:53 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:40362
I20260809 12:03:53.876161 6846 tablet_service.cc:1514] Processing DeleteTablet for tablet 4049bcbc32544e168b81ae8e3e2f3113 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:53 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:58640
I20260809 12:03:53.876658 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 6573
I20260809 12:03:53.880806 7187 tablet_replica.cc:330] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: stopping tablet replica
I20260809 12:03:53.881202 7184 tablet_replica.cc:330] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: stopping tablet replica
I20260809 12:03:53.881644 7186 tablet_replica.cc:330] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: stopping tablet replica
I20260809 12:03:53.882105 7186 raft_consensus.cc:2229] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:53.882304 7187 raft_consensus.cc:2229] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:53.882680 7186 raft_consensus.cc:2258] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:53.882161 7184 raft_consensus.cc:2229] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:03:53.883834 7184 raft_consensus.cc:2258] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:53.884078 7187 raft_consensus.cc:2258] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:53.885073 7186 ts_tablet_manager.cc:1905] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:53.886390 7187 ts_tablet_manager.cc:1905] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:53.888605 7184 ts_tablet_manager.cc:1905] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:53.892561 7186 ts_tablet_manager.cc:1918] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:03:53.892805 7186 log.cc:1196] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/4049bcbc32544e168b81ae8e3e2f3113
I20260809 12:03:53.893416 7186 ts_tablet_manager.cc:1939] T 4049bcbc32544e168b81ae8e3e2f3113 P 0f79fe5013264cd28bec6eb7718c8e94: Deleting consensus metadata
I20260809 12:03:53.894317 7187 ts_tablet_manager.cc:1918] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:03:53.894534 7187 log.cc:1196] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/4049bcbc32544e168b81ae8e3e2f3113
I20260809 12:03:53.895123 7187 ts_tablet_manager.cc:1939] T 4049bcbc32544e168b81ae8e3e2f3113 P 92d3a6d58ae84f908ee5134bc42c3265: Deleting consensus metadata
I20260809 12:03:53.895411 6468 catalog_manager.cc:4841] TS 0f79fe5013264cd28bec6eb7718c8e94 (127.0.147.67:42013): tablet 4049bcbc32544e168b81ae8e3e2f3113 (table test-table [id=48dadab510bc480aad4a6d23b4ae3b18]) successfully deleted
I20260809 12:03:53.896106 7184 ts_tablet_manager.cc:1918] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:03:53.896292 7184 log.cc:1196] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/4049bcbc32544e168b81ae8e3e2f3113
I20260809 12:03:53.896790 7184 ts_tablet_manager.cc:1939] T 4049bcbc32544e168b81ae8e3e2f3113 P 0cf567d59a2b434798fe9786884e57d2: Deleting consensus metadata
I20260809 12:03:53.896962 6468 catalog_manager.cc:4841] TS 92d3a6d58ae84f908ee5134bc42c3265 (127.0.147.66:36503): tablet 4049bcbc32544e168b81ae8e3e2f3113 (table test-table [id=48dadab510bc480aad4a6d23b4ae3b18]) successfully deleted
Aug 09 12:03:53 dist-test-slave-sb2z krb5kdc[6427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:03:53.898238 6468 catalog_manager.cc:4841] TS 0cf567d59a2b434798fe9786884e57d2 (127.0.147.65:37553): tablet 4049bcbc32544e168b81ae8e3e2f3113 (table test-table [id=48dadab510bc480aad4a6d23b4ae3b18]) successfully deleted
I20260809 12:03:53.905560 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:53.877880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.0.147.65:37553)
0809 12:03:53.878046 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:53.878052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:53.881037 (+ 2985us) server_negotiation.cc:407] Connection header received
0809 12:03:53.881239 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:53.881244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:53.881323 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:53.881425 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:53.883332 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.884661 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.886544 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:53.887707 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:53.887834 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:53.898957 (+ 11123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:53.898968 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:53.898982 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:53.899026 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:53.902381 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:53.903106 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:53.903116 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:53.903119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:53.903267 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:53.903739 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:53.903746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:53.903749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:53.904096 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:53.904426 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:53.904760 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 12:03:53.905025 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:03:53.906227 6745 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:54.022272 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 6573
I20260809 12:03:54.049297 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 6762
Aug 09 12:03:54 dist-test-slave-sb2z krb5kdc[6427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:03:54.065079 7135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:54.050137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.0.147.66:36503)
0809 12:03:54.050347 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:54.050353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:54.050376 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:03:54.050467 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:54.050474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:54.050541 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:54.050650 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:54.052028 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:54.052772 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:54.054016 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:54.054424 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:54.054561 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:54.059218 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:54.059232 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:54.059248 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:54.059283 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:54.062120 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:54.062834 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:54.062842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:54.062845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:54.062988 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:54.063385 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:54.063391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:54.063394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:54.063867 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:54.064186 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:54.064499 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 12:03:54.064726 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:03:54.065752 6927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:54.155637 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 6762
I20260809 12:03:54.182171 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 6942
Aug 09 12:03:54 dist-test-slave-sb2z krb5kdc[6427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:03:54.196942 7138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:54.183002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40374 (local address 127.0.147.67:42013)
0809 12:03:54.183201 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:54.183206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:54.183226 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:03:54.183381 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:54.183385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:54.183450 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:54.183578 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:54.184849 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:54.185547 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:54.186753 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:54.187136 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:54.187224 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:54.191606 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:54.191625 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:54.191641 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:03:54.191679 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:54.194488 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:54.195045 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:54.195053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:54.195055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:54.195194 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:03:54.195506 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:03:54.195509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:03:54.195511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:03:54.195941 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:54.196168 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:54.196415 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 12:03:54.196578 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:03:54.197573 7106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:54.287683 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 6942
I20260809 12:03:54.313743 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 6445
I20260809 12:03:54.314826 6529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:54.426838 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 6445
2026-08-09T12:03:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4555 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T12:03:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:54Z Disabled control of system clock
I20260809 12:03:54.514773 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:37749
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:37749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 12:03:54.844256 7199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:54.844578 7199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:54.844727 7199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:54.852041 7199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:54.852164 7199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:54.852246 7199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:54.852355 7199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:54.852444 7199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:03:54.862062 7199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:37749
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:37749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:54.863417 7199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:54.865382 7199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:54.877635 7209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:54.877653 7207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:54.877730 7206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:54.878574 7199 server_base.cc:1034] running on GCE node
I20260809 12:03:54.879443 7199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:54.880817 7199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:54.882156 7199 hybrid_clock.cc:648] HybridClock initialized: now 1786277034882121 us; error 41 us; skew 500 ppm
I20260809 12:03:54.882532 7199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:03:54.884860 7199 webserver.cc:460] Webserver started at http://127.0.147.126:35657/ using document root <none> and password file <none>
I20260809 12:03:54.885565 7199 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:54.885703 7199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:54.886055 7199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:54.888996 7199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "7d26f5d6221d42c685a30cbab5f62e32"
format_stamp: "Formatted at 2026-08-09 12:03:54 on dist-test-slave-sb2z"
server_key: "bbe87e10a715f81ab0f82393774953f9"
server_key_iv: "e0ea612ea2b19145984039178444ae25"
server_key_version: "encryptionkey@0"
I20260809 12:03:54.889838 7199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "7d26f5d6221d42c685a30cbab5f62e32"
format_stamp: "Formatted at 2026-08-09 12:03:54 on dist-test-slave-sb2z"
server_key: "bbe87e10a715f81ab0f82393774953f9"
server_key_iv: "e0ea612ea2b19145984039178444ae25"
server_key_version: "encryptionkey@0"
I20260809 12:03:54.895035 7199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:03:54.898432 7216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:54.899730 7199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:54.899958 7199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "7d26f5d6221d42c685a30cbab5f62e32"
format_stamp: "Formatted at 2026-08-09 12:03:54 on dist-test-slave-sb2z"
server_key: "bbe87e10a715f81ab0f82393774953f9"
server_key_iv: "e0ea612ea2b19145984039178444ae25"
server_key_version: "encryptionkey@0"
I20260809 12:03:54.900197 7199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:54.905947 7199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:54.906723 7199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:54.907043 7199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:54.925993 7199 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:37749
I20260809 12:03:54.926004 7289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:37749 every 8 connection(s)
I20260809 12:03:54.927356 7199 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:54.931115 7291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:54.935649 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 7199
I20260809 12:03:54.935840 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:54.936546 589 external_mini_cluster.cc:1428] Setting key 91c2543a8d3fd2309ad209b95d6379d3
I20260809 12:03:54.943513 7291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32: Bootstrap starting.
I20260809 12:03:54.947345 7291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:54.947386 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:54.939369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47562 (local address 127.0.147.126:37749)
0809 12:03:54.940626 (+ 1257us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:54.940641 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:54.940701 (+ 60us) server_negotiation.cc:407] Connection header received
0809 12:03:54.942959 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:54.942984 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:54.943487 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:54.943934 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:54.945554 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:54.945564 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:54.945586 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:54.945605 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:54.945686 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:54.945708 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:54.945933 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:54.946184 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 12:03:54.946457 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":465,"threads_started":1}
I20260809 12:03:54.948817 7291 log.cc:824] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:54.951500 7291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32: No bootstrap required, opened a new log
I20260809 12:03:54.958391 7291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d26f5d6221d42c685a30cbab5f62e32" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 37749 } }
I20260809 12:03:54.958746 7291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:54.958854 7291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d26f5d6221d42c685a30cbab5f62e32, State: Initialized, Role: FOLLOWER
I20260809 12:03:54.959408 7291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d26f5d6221d42c685a30cbab5f62e32" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 37749 } }
I20260809 12:03:54.959720 7291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:54.959882 7291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:54.960065 7291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:54.962203 7291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d26f5d6221d42c685a30cbab5f62e32" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 37749 } }
I20260809 12:03:54.962692 7291 leader_election.cc:302] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d26f5d6221d42c685a30cbab5f62e32; no voters:
I20260809 12:03:54.963263 7291 leader_election.cc:288] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:54.963403 7309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:54.964190 7309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [term 1 LEADER]: Becoming Leader. State: Replica: 7d26f5d6221d42c685a30cbab5f62e32, State: Running, Role: LEADER
I20260809 12:03:54.964780 7309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d26f5d6221d42c685a30cbab5f62e32" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 37749 } }
I20260809 12:03:54.965363 7291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:54.967628 7312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d26f5d6221d42c685a30cbab5f62e32. Latest consensus state: current_term: 1 leader_uuid: "7d26f5d6221d42c685a30cbab5f62e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d26f5d6221d42c685a30cbab5f62e32" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 37749 } } }
I20260809 12:03:54.967603 7311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d26f5d6221d42c685a30cbab5f62e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d26f5d6221d42c685a30cbab5f62e32" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 37749 } } }
I20260809 12:03:54.968295 7311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:54.968302 7312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:54.969985 7319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:54.975011 7319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:54.984246 7319 catalog_manager.cc:1347] Generated new cluster ID: 0f6032ae6a5c4384a82476d647109ea4
I20260809 12:03:54.984396 7319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:55.025122 7319 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:55.025568 7319 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:55.036325 7319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32: Generated new TSK 0
I20260809 12:03:55.037317 7319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:03:55.052886 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:37749
--builtin_ntp_servers=127.0.147.84:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 12:03:55.396994 7340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:55.397281 7340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:55.397377 7340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:55.404933 7340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:55.405040 7340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:55.405148 7340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:03:55.414549 7340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:37749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:55.415783 7340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:55.417502 7340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:55.430989 7349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:55.431067 7352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:55.431149 7350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:55.432102 7340 server_base.cc:1034] running on GCE node
I20260809 12:03:55.432732 7340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:55.433744 7340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:55.434948 7340 hybrid_clock.cc:648] HybridClock initialized: now 1786277035434898 us; error 37 us; skew 500 ppm
I20260809 12:03:55.435289 7340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:03:55.437606 7340 webserver.cc:460] Webserver started at http://127.0.147.65:39837/ using document root <none> and password file <none>
I20260809 12:03:55.438300 7340 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:55.438474 7340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:55.438804 7340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:55.441766 7340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "b8d2b23b5e16483f9967d1165a2f6418"
format_stamp: "Formatted at 2026-08-09 12:03:55 on dist-test-slave-sb2z"
server_key: "6e855dfd22083d8a36691bb27271fc4b"
server_key_iv: "562e56bfa84e080d97e2cb30bfb61852"
server_key_version: "encryptionkey@0"
I20260809 12:03:55.442602 7340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "b8d2b23b5e16483f9967d1165a2f6418"
format_stamp: "Formatted at 2026-08-09 12:03:55 on dist-test-slave-sb2z"
server_key: "6e855dfd22083d8a36691bb27271fc4b"
server_key_iv: "562e56bfa84e080d97e2cb30bfb61852"
server_key_version: "encryptionkey@0"
I20260809 12:03:55.448127 7340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 12:03:55.451565 7363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:55.452880 7340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:03:55.453079 7340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "b8d2b23b5e16483f9967d1165a2f6418"
format_stamp: "Formatted at 2026-08-09 12:03:55 on dist-test-slave-sb2z"
server_key: "6e855dfd22083d8a36691bb27271fc4b"
server_key_iv: "562e56bfa84e080d97e2cb30bfb61852"
server_key_version: "encryptionkey@0"
I20260809 12:03:55.453317 7340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:55.459115 7340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:55.459985 7340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:55.460279 7340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:55.461223 7340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:55.462622 7340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:55.462746 7340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:55.462886 7340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:55.462976 7340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:55.497999 7340 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:40485
I20260809 12:03:55.498103 7521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:40485 every 8 connection(s)
I20260809 12:03:55.499421 7340 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:03:55.499831 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 7340
I20260809 12:03:55.500065 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:03:55.500975 589 external_mini_cluster.cc:1428] Setting key 44af77d7082217a01c433198585bd661
I20260809 12:03:55.504710 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:37749
--builtin_ntp_servers=127.0.147.84:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 12:03:55.509912 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:55.502411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:47529 (local address 127.0.147.126:37749)
0809 12:03:55.502875 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:55.502879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:55.504741 (+ 1862us) server_negotiation.cc:407] Connection header received
0809 12:03:55.507498 (+ 2757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:55.507525 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:55.507634 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:55.507768 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:55.509236 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:55.509241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:55.509244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:55.509261 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:55.509305 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:55.509317 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:55.509412 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:55.509633 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 12:03:55.509684 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":256,"threads_started":1}
I20260809 12:03:55.511215 7526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:55.503169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:37749 (local address 127.0.147.65:47529)
0809 12:03:55.504551 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:55.504609 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:55.507186 (+ 2577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:55.508007 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:55.508020 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:55.508151 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:55.508873 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:03:55.508879 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:55.508931 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:03:55.509044 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:55.509417 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:55.509430 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:55.509440 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:03:55.509713 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 12:03:55.509970 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":449,"threads_started":1}
I20260809 12:03:55.512800 7522 heartbeater.cc:347] Connected to a master server at 127.0.147.126:37749
I20260809 12:03:55.513114 7522 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:55.513690 7522 heartbeater.cc:511] Master 127.0.147.126:37749 requested a full tablet report, sending...
I20260809 12:03:55.515995 7244 ts_manager.cc:194] Registered new tserver with Master: b8d2b23b5e16483f9967d1165a2f6418 (127.0.147.65:40485)
W20260809 12:03:55.861841 7527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:55.862125 7527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:55.862304 7527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:55.870074 7527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:55.870172 7527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:55.870383 7527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:03:55.879676 7527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:37749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:55.881028 7527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:55.882762 7527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:55.895810 7534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:55.895846 7535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:55.896277 7538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:55.896787 7527 server_base.cc:1034] running on GCE node
I20260809 12:03:55.897343 7527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:55.898180 7527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:55.899375 7527 hybrid_clock.cc:648] HybridClock initialized: now 1786277035899337 us; error 36 us; skew 500 ppm
I20260809 12:03:55.899734 7527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:03:55.901957 7527 webserver.cc:460] Webserver started at http://127.0.147.66:35997/ using document root <none> and password file <none>
I20260809 12:03:55.902621 7527 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:55.902737 7527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:55.903061 7527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:55.905964 7527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "a51db210f5a94d3c851336e815b719e8"
format_stamp: "Formatted at 2026-08-09 12:03:55 on dist-test-slave-sb2z"
server_key: "e98e846abf78fb1908f4543fa5075d49"
server_key_iv: "20946d1a54f8255c369dbaf73f5807f5"
server_key_version: "encryptionkey@0"
I20260809 12:03:55.906769 7527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "a51db210f5a94d3c851336e815b719e8"
format_stamp: "Formatted at 2026-08-09 12:03:55 on dist-test-slave-sb2z"
server_key: "e98e846abf78fb1908f4543fa5075d49"
server_key_iv: "20946d1a54f8255c369dbaf73f5807f5"
server_key_version: "encryptionkey@0"
I20260809 12:03:55.912011 7527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 12:03:55.915290 7548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:55.916581 7527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:55.916766 7527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "a51db210f5a94d3c851336e815b719e8"
format_stamp: "Formatted at 2026-08-09 12:03:55 on dist-test-slave-sb2z"
server_key: "e98e846abf78fb1908f4543fa5075d49"
server_key_iv: "20946d1a54f8255c369dbaf73f5807f5"
server_key_version: "encryptionkey@0"
I20260809 12:03:55.916997 7527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:55.922537 7527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:55.923311 7527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:55.923691 7527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:55.924587 7527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:55.926028 7527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:55.926151 7527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:55.926301 7527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:55.926393 7527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:55.964102 7527 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:40249
I20260809 12:03:55.964191 7701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:40249 every 8 connection(s)
I20260809 12:03:55.965543 7527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:03:55.972005 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 7527
I20260809 12:03:55.972182 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:03:55.972770 589 external_mini_cluster.cc:1428] Setting key c3a4ae409552d13322de7e158f2d7763
I20260809 12:03:55.974042 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:55.968210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:53229 (local address 127.0.147.126:37749)
0809 12:03:55.968422 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:55.968430 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:55.969882 (+ 1452us) server_negotiation.cc:407] Connection header received
0809 12:03:55.971516 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:55.971555 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:55.971634 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:55.971722 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:55.973303 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:55.973308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:55.973311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:55.973326 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:55.973357 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:55.973368 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:55.973467 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:55.973722 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 12:03:55.973791 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:03:55.975132 7706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:55.968779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:37749 (local address 127.0.147.66:53229)
0809 12:03:55.969729 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:55.969764 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:55.971281 (+ 1517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:55.972034 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:55.972046 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:55.972169 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:55.972937 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:03:55.972941 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:55.972991 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:03:55.973114 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:55.973474 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:55.973487 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:55.973495 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:03:55.973773 (+ 278us) client_negotiation.cc:240] Negotiation successful
0809 12:03:55.974020 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":345,"threads_started":1}
I20260809 12:03:55.976105 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:37749
--builtin_ntp_servers=127.0.147.84:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 12:03:55.976606 7702 heartbeater.cc:347] Connected to a master server at 127.0.147.126:37749
I20260809 12:03:55.976881 7702 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:55.977422 7702 heartbeater.cc:511] Master 127.0.147.126:37749 requested a full tablet report, sending...
I20260809 12:03:55.978844 7244 ts_manager.cc:194] Registered new tserver with Master: a51db210f5a94d3c851336e815b719e8 (127.0.147.66:40249)
W20260809 12:03:56.316143 7709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:56.316440 7709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:56.316650 7709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:56.323944 7709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:56.324039 7709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:56.324226 7709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:03:56.333046 7709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:37749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:56.334219 7709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:56.336004 7709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:56.349153 7720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:56.349120 7726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:56.349233 7721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:56.357060 7709 server_base.cc:1034] running on GCE node
I20260809 12:03:56.357738 7709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:56.358731 7709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:56.359959 7709 hybrid_clock.cc:648] HybridClock initialized: now 1786277036359899 us; error 49 us; skew 500 ppm
I20260809 12:03:56.360277 7709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:03:56.362521 7709 webserver.cc:460] Webserver started at http://127.0.147.67:42227/ using document root <none> and password file <none>
I20260809 12:03:56.363194 7709 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:56.363329 7709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:56.363730 7709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:56.366792 7709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "d4df3cede5434be990d6a7e4b54ebf60"
format_stamp: "Formatted at 2026-08-09 12:03:56 on dist-test-slave-sb2z"
server_key: "27c0222a10490c8d1e75e2fcea218d79"
server_key_iv: "91dbf6e7fd17594c4b7a69476f01dcff"
server_key_version: "encryptionkey@0"
I20260809 12:03:56.367636 7709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "d4df3cede5434be990d6a7e4b54ebf60"
format_stamp: "Formatted at 2026-08-09 12:03:56 on dist-test-slave-sb2z"
server_key: "27c0222a10490c8d1e75e2fcea218d79"
server_key_iv: "91dbf6e7fd17594c4b7a69476f01dcff"
server_key_version: "encryptionkey@0"
I20260809 12:03:56.372644 7709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:03:56.376031 7735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:56.377339 7709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:03:56.377518 7709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "d4df3cede5434be990d6a7e4b54ebf60"
format_stamp: "Formatted at 2026-08-09 12:03:56 on dist-test-slave-sb2z"
server_key: "27c0222a10490c8d1e75e2fcea218d79"
server_key_iv: "91dbf6e7fd17594c4b7a69476f01dcff"
server_key_version: "encryptionkey@0"
I20260809 12:03:56.377767 7709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:56.383491 7709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:56.384316 7709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:56.384598 7709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:56.385576 7709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:03:56.386935 7709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:03:56.387059 7709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:56.387197 7709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:03:56.387285 7709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:56.422092 7709 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:43405
I20260809 12:03:56.422152 7893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:43405 every 8 connection(s)
I20260809 12:03:56.423429 7709 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:03:56.431190 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 7709
I20260809 12:03:56.431383 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:03:56.431998 589 external_mini_cluster.cc:1428] Setting key 0dea08003a6326a7345fc8d6c00ba753
I20260809 12:03:56.432310 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.426061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:39995 (local address 127.0.147.126:37749)
0809 12:03:56.426281 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.426289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.427664 (+ 1375us) server_negotiation.cc:407] Connection header received
0809 12:03:56.429947 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.429952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.430012 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.430103 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.431500 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.431506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.431510 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.431532 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.431599 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.431619 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.431720 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.431965 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.432013 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 12:03:56.433511 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.426583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:37749 (local address 127.0.147.67:39995)
0809 12:03:56.427461 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.427502 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:56.429637 (+ 2135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:56.430305 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:56.430312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:56.430395 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:56.431061 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:03:56.431064 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:56.431115 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:03:56.431242 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:56.431723 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:56.431734 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:56.431742 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:03:56.432074 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 12:03:56.432326 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":271,"threads_started":1}
I20260809 12:03:56.434875 7894 heartbeater.cc:347] Connected to a master server at 127.0.147.126:37749
I20260809 12:03:56.435207 7894 heartbeater.cc:464] Registering TS with master...
I20260809 12:03:56.435840 7894 heartbeater.cc:511] Master 127.0.147.126:37749 requested a full tablet report, sending...
I20260809 12:03:56.437156 7244 ts_manager.cc:194] Registered new tserver with Master: d4df3cede5434be990d6a7e4b54ebf60 (127.0.147.67:43405)
I20260809 12:03:56.447067 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:03:56.454030 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.451738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47566 (local address 127.0.147.126:37749)
0809 12:03:56.451945 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.451954 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.452392 (+ 438us) server_negotiation.cc:407] Connection header received
0809 12:03:56.452500 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.452504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.452549 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.452630 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.453454 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.453458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.453462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.453476 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.453507 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.453518 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.453628 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.453681 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.453724 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:03:56.457928 7244 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:03:56.464891 7244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:03:56.483508 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.477818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40485 (local address 127.0.0.1:50710)
0809 12:03:56.478911 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.478934 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:56.479050 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:56.481237 (+ 2187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:56.481242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:56.481270 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:56.482193 (+ 923us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:03:56.482198 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:56.482234 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:03:56.482339 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:56.482805 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:56.482817 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:56.482824 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 12:03:56.483000 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 12:03:56.483090 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":4464,"threads_started":1}
I20260809 12:03:56.483493 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.478366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.0.147.65:40485)
0809 12:03:56.480504 (+ 2138us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.480513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.480540 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:03:56.480610 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.480619 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.480839 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.481083 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.482554 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.482572 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.482587 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.482607 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.482673 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.482698 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.482849 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.482985 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.483087 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1967,"thread_start_us":1832,"threads_started":1}
I20260809 12:03:56.484290 7909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.477128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:40249 (local address 127.0.0.1:54586)
0809 12:03:56.477931 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.477975 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:56.478127 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:56.481010 (+ 2883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:56.481031 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:56.481161 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:56.482359 (+ 1198us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:03:56.482363 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:56.482379 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:03:56.482429 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:56.483515 (+ 1086us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:56.483520 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:56.483563 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 12:03:56.483694 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 12:03:56.483748 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":343,"threads_started":1}
I20260809 12:03:56.485280 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.477277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54586 (local address 127.0.147.66:40249)
0809 12:03:56.480135 (+ 2858us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.480145 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.480184 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:03:56.480259 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.480267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.480667 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.481937 (+ 1270us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.483248 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.483256 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.483269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.483285 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.483339 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.483365 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.484784 (+ 1419us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.484851 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.484955 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2644,"thread_start_us":2509,"threads_started":1}
I20260809 12:03:56.486550 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.478893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59482 (local address 127.0.147.67:43405)
0809 12:03:56.479874 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.479882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.482613 (+ 2731us) server_negotiation.cc:407] Connection header received
0809 12:03:56.482709 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.482719 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.482922 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.483086 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.485691 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.485701 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.485713 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.485732 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.485788 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.485808 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.485983 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.486069 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.486162 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":531,"threads_started":1}
I20260809 12:03:56.486591 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.478289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43405 (local address 127.0.0.1:59482)
0809 12:03:56.482482 (+ 4193us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.482499 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:56.482601 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:56.483767 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:56.483774 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:56.483802 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:56.485440 (+ 1638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:03:56.485443 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:56.485460 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:03:56.485503 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:56.485933 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:56.485937 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:56.485944 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 12:03:56.486083 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 12:03:56.486138 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4039,"thread_start_us":1003,"threads_started":1}
I20260809 12:03:56.487998 7431 tablet_service.cc:1467] Processing CreateTablet for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d (DEFAULT_TABLE table=test-table [id=519a1ede950e4083b07ea1255ec30674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:56.488859 7622 tablet_service.cc:1467] Processing CreateTablet for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d (DEFAULT_TABLE table=test-table [id=519a1ede950e4083b07ea1255ec30674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:56.489509 7431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:56.490171 7622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:56.490105 7798 tablet_service.cc:1467] Processing CreateTablet for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d (DEFAULT_TABLE table=test-table [id=519a1ede950e4083b07ea1255ec30674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:03:56.491508 7798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:56.508177 7917 tablet_bootstrap.cc:492] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418: Bootstrap starting.
I20260809 12:03:56.511081 7918 tablet_bootstrap.cc:492] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: Bootstrap starting.
I20260809 12:03:56.514249 7917 tablet_bootstrap.cc:654] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:56.515151 7919 tablet_bootstrap.cc:492] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: Bootstrap starting.
I20260809 12:03:56.515692 7917 log.cc:824] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:56.516757 7918 tablet_bootstrap.cc:654] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:56.517830 7918 log.cc:824] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:56.519143 7917 tablet_bootstrap.cc:492] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418: No bootstrap required, opened a new log
I20260809 12:03:56.519167 7522 heartbeater.cc:503] Master 127.0.147.126:37749 was elected leader, sending a full tablet report...
I20260809 12:03:56.519762 7919 tablet_bootstrap.cc:654] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:56.520777 7917 ts_tablet_manager.cc:1397] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.006s
I20260809 12:03:56.522113 7918 tablet_bootstrap.cc:492] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: No bootstrap required, opened a new log
I20260809 12:03:56.522369 7918 ts_tablet_manager.cc:1397] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.004s
I20260809 12:03:56.523368 7919 log.cc:824] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:56.526702 7919 tablet_bootstrap.cc:492] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: No bootstrap required, opened a new log
I20260809 12:03:56.526974 7919 ts_tablet_manager.cc:1397] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.002s
I20260809 12:03:56.530335 7918 raft_consensus.cc:348] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } }
I20260809 12:03:56.530704 7918 raft_consensus.cc:374] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:56.530807 7918 raft_consensus.cc:729] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51db210f5a94d3c851336e815b719e8, State: Initialized, Role: FOLLOWER
I20260809 12:03:56.531414 7918 consensus_queue.cc:260] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } }
I20260809 12:03:56.531464 7917 raft_consensus.cc:348] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } }
I20260809 12:03:56.531913 7917 raft_consensus.cc:374] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:56.532112 7917 raft_consensus.cc:729] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d2b23b5e16483f9967d1165a2f6418, State: Initialized, Role: FOLLOWER
I20260809 12:03:56.532562 7702 heartbeater.cc:503] Master 127.0.147.126:37749 was elected leader, sending a full tablet report...
I20260809 12:03:56.532793 7917 consensus_queue.cc:260] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } }
I20260809 12:03:56.533277 7918 ts_tablet_manager.cc:1428] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: Time spent starting tablet: real 0.011s user 0.001s sys 0.010s
I20260809 12:03:56.539740 7919 raft_consensus.cc:348] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } }
I20260809 12:03:56.540223 7919 raft_consensus.cc:374] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:56.540336 7919 raft_consensus.cc:729] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4df3cede5434be990d6a7e4b54ebf60, State: Initialized, Role: FOLLOWER
I20260809 12:03:56.541029 7917 ts_tablet_manager.cc:1428] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418: Time spent starting tablet: real 0.020s user 0.012s sys 0.005s
I20260809 12:03:56.541046 7919 consensus_queue.cc:260] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } }
I20260809 12:03:56.542291 7894 heartbeater.cc:503] Master 127.0.147.126:37749 was elected leader, sending a full tablet report...
I20260809 12:03:56.543025 7919 ts_tablet_manager.cc:1428] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260809 12:03:56.586289 7931 raft_consensus.cc:482] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:03:56.586669 7931 raft_consensus.cc:504] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } }
I20260809 12:03:56.588369 7931 leader_election.cc:288] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a51db210f5a94d3c851336e815b719e8 (127.0.147.66:40249), d4df3cede5434be990d6a7e4b54ebf60 (127.0.147.67:43405)
I20260809 12:03:56.591838 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.588839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:60649 (local address 127.0.147.66:40249)
0809 12:03:56.589080 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.589090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.589563 (+ 473us) server_negotiation.cc:407] Connection header received
0809 12:03:56.589744 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.589750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.589822 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.589918 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.591124 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.591129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.591134 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.591152 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.591195 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.591207 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.591329 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.591433 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.591502 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 12:03:56.592101 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.588756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:40249 (local address 127.0.147.65:60649)
0809 12:03:56.589422 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.589438 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:56.589597 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:56.590058 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:56.590065 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:56.590094 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:56.590891 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:03:56.590894 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:56.590910 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:03:56.590956 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:56.591303 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:56.591308 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:56.591313 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:03:56.591467 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 12:03:56.591547 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":384,"threads_started":1}
I20260809 12:03:56.592525 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.589717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:41431 (local address 127.0.147.67:43405)
0809 12:03:56.589934 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.589940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.590463 (+ 523us) server_negotiation.cc:407] Connection header received
0809 12:03:56.590645 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.590649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.590695 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.590797 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.591790 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.591796 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.591800 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.591825 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.591858 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.591869 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.592009 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.592191 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.592254 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:03:56.592679 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.589570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43405 (local address 127.0.147.65:41431)
0809 12:03:56.590277 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.590297 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:56.590497 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:56.590822 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:56.590826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:56.590854 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:03:56.591594 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:03:56.591597 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:03:56.591609 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:03:56.591644 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:03:56.592073 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:03:56.592077 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:03:56.592082 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:03:56.592231 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 12:03:56.592278 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":388,"threads_started":1}
I20260809 12:03:56.593151 7647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e5dfe4ba6394f0ab19bf5631e9e516d" candidate_uuid: "b8d2b23b5e16483f9967d1165a2f6418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a51db210f5a94d3c851336e815b719e8" is_pre_election: true
I20260809 12:03:56.593645 7647 raft_consensus.cc:2454] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d2b23b5e16483f9967d1165a2f6418 in term 0.
I20260809 12:03:56.593677 7827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e5dfe4ba6394f0ab19bf5631e9e516d" candidate_uuid: "b8d2b23b5e16483f9967d1165a2f6418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4df3cede5434be990d6a7e4b54ebf60" is_pre_election: true
I20260809 12:03:56.594215 7827 raft_consensus.cc:2454] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d2b23b5e16483f9967d1165a2f6418 in term 0.
I20260809 12:03:56.594341 7365 leader_election.cc:302] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a51db210f5a94d3c851336e815b719e8, b8d2b23b5e16483f9967d1165a2f6418; no voters:
I20260809 12:03:56.594751 7931 raft_consensus.cc:2781] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:03:56.594937 7931 raft_consensus.cc:482] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:03:56.595069 7931 raft_consensus.cc:3037] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:56.597340 7931 raft_consensus.cc:504] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } }
I20260809 12:03:56.597922 7931 leader_election.cc:288] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [CANDIDATE]: Term 1 election: Requested vote from peers a51db210f5a94d3c851336e815b719e8 (127.0.147.66:40249), d4df3cede5434be990d6a7e4b54ebf60 (127.0.147.67:43405)
I20260809 12:03:56.598413 7647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e5dfe4ba6394f0ab19bf5631e9e516d" candidate_uuid: "b8d2b23b5e16483f9967d1165a2f6418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a51db210f5a94d3c851336e815b719e8"
I20260809 12:03:56.598474 7827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e5dfe4ba6394f0ab19bf5631e9e516d" candidate_uuid: "b8d2b23b5e16483f9967d1165a2f6418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4df3cede5434be990d6a7e4b54ebf60"
I20260809 12:03:56.598595 7647 raft_consensus.cc:3037] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:56.598690 7827 raft_consensus.cc:3037] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:56.600963 7647 raft_consensus.cc:2454] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d2b23b5e16483f9967d1165a2f6418 in term 1.
I20260809 12:03:56.600993 7827 raft_consensus.cc:2454] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d2b23b5e16483f9967d1165a2f6418 in term 1.
I20260809 12:03:56.601354 7365 leader_election.cc:302] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a51db210f5a94d3c851336e815b719e8, b8d2b23b5e16483f9967d1165a2f6418; no voters:
I20260809 12:03:56.601632 7931 raft_consensus.cc:2781] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:56.602159 7931 raft_consensus.cc:686] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 1 LEADER]: Becoming Leader. State: Replica: b8d2b23b5e16483f9967d1165a2f6418, State: Running, Role: LEADER
I20260809 12:03:56.602773 7931 consensus_queue.cc:237] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } }
I20260809 12:03:56.607975 7244 catalog_manager.cc:5495] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8d2b23b5e16483f9967d1165a2f6418 (127.0.147.65). New cstate: current_term: 1 leader_uuid: "b8d2b23b5e16483f9967d1165a2f6418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8d2b23b5e16483f9967d1165a2f6418" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40485 } health_report { overall_health: HEALTHY } } }
I20260809 12:03:56.642122 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.639667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.0.147.65:40485)
0809 12:03:56.639870 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.639876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.639997 (+ 121us) server_negotiation.cc:407] Connection header received
0809 12:03:56.640252 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.640256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.640334 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.640444 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.641453 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.641459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.641464 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.641481 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.641521 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.641533 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.641661 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.641794 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.641853 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:03:56.650398 7827 raft_consensus.cc:1261] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [term 1 FOLLOWER]: Refusing update from remote peer b8d2b23b5e16483f9967d1165a2f6418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:56.650398 7647 raft_consensus.cc:1261] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [term 1 FOLLOWER]: Refusing update from remote peer b8d2b23b5e16483f9967d1165a2f6418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:03:56.651362 7936 consensus_queue.cc:1035] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4df3cede5434be990d6a7e4b54ebf60" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:56.651818 7931 consensus_queue.cc:1035] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a51db210f5a94d3c851336e815b719e8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 40249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:03:56.668387 7942 mvcc.cc:205] Tried to move back new op lower bound from 7316590742109143040 to 7316590741929730048. Current Snapshot: MvccSnapshot[applied={T|T < 7316590742109143040}]
I20260809 12:03:56.678821 7243 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:56.679077 7243 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:03:56.682979 7243 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d26f5d6221d42c685a30cbab5f62e32: Sending DeleteTablet for 3 replicas of tablet 3e5dfe4ba6394f0ab19bf5631e9e516d
I20260809 12:03:56.684181 7622 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:56 UTC) from {username='slave'} at 127.0.0.1:54586
I20260809 12:03:56.684315 7798 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:56 UTC) from {username='slave'} at 127.0.0.1:59482
I20260809 12:03:56.684337 7431 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:03:56 UTC) from {username='slave'} at 127.0.0.1:50710
I20260809 12:03:56.686770 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 7340
I20260809 12:03:56.695670 7954 tablet_replica.cc:330] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: stopping tablet replica
I20260809 12:03:56.696245 7954 raft_consensus.cc:2229] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:56.696848 7954 raft_consensus.cc:2258] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:56.697876 7956 tablet_replica.cc:330] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: stopping tablet replica
I20260809 12:03:56.699396 7954 ts_tablet_manager.cc:1905] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:56.700349 7957 tablet_replica.cc:330] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418: stopping tablet replica
I20260809 12:03:56.700754 7957 raft_consensus.cc:2229] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:03:56.701455 7957 raft_consensus.cc:2258] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:56.702483 7956 raft_consensus.cc:2229] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:03:56.703024 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.688647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.0.147.65:40485)
0809 12:03:56.688873 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.688882 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.698537 (+ 9655us) server_negotiation.cc:407] Connection header received
0809 12:03:56.698894 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.698903 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.699009 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.699102 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.702151 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.702158 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.702162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.702180 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.702218 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.702235 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.702382 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.702623 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.702693 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:03:56.703006 7956 raft_consensus.cc:2258] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:03:56.704344 7513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:56.705500 7956 ts_tablet_manager.cc:1905] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:56.706737 7954 ts_tablet_manager.cc:1918] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:03:56.706975 7954 log.cc:1196] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/3e5dfe4ba6394f0ab19bf5631e9e516d
I20260809 12:03:56.707749 7954 ts_tablet_manager.cc:1939] T 3e5dfe4ba6394f0ab19bf5631e9e516d P a51db210f5a94d3c851336e815b719e8: Deleting consensus metadata
I20260809 12:03:56.710017 7219 catalog_manager.cc:4841] TS a51db210f5a94d3c851336e815b719e8 (127.0.147.66:40249): tablet 3e5dfe4ba6394f0ab19bf5631e9e516d (table test-table [id=519a1ede950e4083b07ea1255ec30674]) successfully deleted
I20260809 12:03:56.713415 7956 ts_tablet_manager.cc:1918] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:03:56.713603 7956 log.cc:1196] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/3e5dfe4ba6394f0ab19bf5631e9e516d
I20260809 12:03:56.714121 7956 ts_tablet_manager.cc:1939] T 3e5dfe4ba6394f0ab19bf5631e9e516d P d4df3cede5434be990d6a7e4b54ebf60: Deleting consensus metadata
I20260809 12:03:56.715796 7219 catalog_manager.cc:4841] TS d4df3cede5434be990d6a7e4b54ebf60 (127.0.147.67:43405): tablet 3e5dfe4ba6394f0ab19bf5631e9e516d (table test-table [id=519a1ede950e4083b07ea1255ec30674]) successfully deleted
I20260809 12:03:56.706293 7957 ts_tablet_manager.cc:1905] T 3e5dfe4ba6394f0ab19bf5631e9e516d P b8d2b23b5e16483f9967d1165a2f6418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:03:56.803133 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 7340
W20260809 12:03:56.828701 7222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.147.65:40485 (error 108)
I20260809 12:03:56.829064 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 7527
I20260809 12:03:56.830876 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.830432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40485 (local address 127.0.0.1:50726)
0809 12:03:56.830614 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.830734 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 12:03:56.831126 7222 catalog_manager.cc:4584] TS b8d2b23b5e16483f9967d1165a2f6418 (127.0.147.65:40485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e5dfe4ba6394f0ab19bf5631e9e516d: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40485: connect: Connection refused (error 111)
I20260809 12:03:56.832271 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.830012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.0.147.66:40249)
0809 12:03:56.830187 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.830192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.830212 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:03:56.830382 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.830388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.830489 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.830599 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.831626 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.831632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.831637 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.831662 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.831692 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.831705 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.831835 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.831928 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.831991 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:03:56.832890 7696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:56.884686 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.884202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40485 (local address 127.0.0.1:50734)
0809 12:03:56.884418 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.884528 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 12:03:56.885749 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.885384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40485 (local address 127.0.0.1:50746)
0809 12:03:56.885561 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.885632 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 12:03:56.921258 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 7527
I20260809 12:03:56.946064 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 7709
I20260809 12:03:56.946365 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.945852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40485 (local address 127.0.0.1:50760)
0809 12:03:56.946092 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.946218 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 12:03:56.947417 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.947050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40485 (local address 127.0.0.1:50762)
0809 12:03:56.947214 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:56.947298 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 12:03:56.949194 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:56.946843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.0.147.67:43405)
0809 12:03:56.947044 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:56.947051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:56.947073 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:03:56.947287 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:56.947293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:56.947387 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:56.947496 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:56.948468 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:56.948477 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:56.948481 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:56.948505 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:56.948549 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:56.948565 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:56.948699 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:56.948808 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 12:03:56.948873 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:03:56.949949 7884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:57.028283 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:57.027771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40485 (local address 127.0.0.1:50768)
0809 12:03:57.028033 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:57.028133 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 12:03:57.029351 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:57.028948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40485 (local address 127.0.0.1:50774)
0809 12:03:57.029171 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:57.029239 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260809 12:03:57.040138 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 7709
I20260809 12:03:57.066763 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 7199
I20260809 12:03:57.068063 7282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:57.176892 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 7199
2026-08-09T12:03:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2747 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 12:03:57.228663 589 mini_oidc.cc:152] Starting JWKS server
I20260809 12:03:57.231630 589 webserver.cc:460] Webserver started at https://127.0.0.1:34451/ using document root <none> and password file <none>
I20260809 12:03:57.231957 589 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 12:03:57.232604 589 webserver.cc:460] Webserver started at http://127.0.0.1:35373/ using document root <none> and password file <none>
2026-08-09T12:03:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:57Z Disabled control of system clock
I20260809 12:03:57.267951 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:44025
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:34883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:44025
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34451/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/testchainca.cert with env {}
W20260809 12:03:57.602414 7974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:57.602738 7974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:57.602909 7974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:57.610647 7974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:57.610749 7974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:57.610838 7974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:57.610921 7974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:57.610984 7974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:03:57.611079 7974 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:03:57.611137 7974 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34451/jwks.json
I20260809 12:03:57.621830 7974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:34883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:44025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:44025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34451/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:57.623154 7974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:57.624563 7974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:57.636260 7979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:57.636293 7980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:57.636485 7984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:57.637277 7974 server_base.cc:1034] running on GCE node
I20260809 12:03:57.638095 7974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:57.639365 7974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:57.640571 7974 hybrid_clock.cc:648] HybridClock initialized: now 1786277037640530 us; error 75 us; skew 500 ppm
I20260809 12:03:57.640937 7974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:03:57.643646 7974 webserver.cc:460] Webserver started at http://127.0.147.126:44999/ using document root <none> and password file <none>
I20260809 12:03:57.644335 7974 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:57.644457 7974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:57.644778 7974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:57.647687 7974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "db4de649d3d842bebb9e46ce80733ac3"
format_stamp: "Formatted at 2026-08-09 12:03:57 on dist-test-slave-sb2z"
server_key: "e24a86bab80e88903fa00636c525a0ba"
server_key_iv: "e35eaa37caa10d84d7d935f9c182a851"
server_key_version: "encryptionkey@0"
I20260809 12:03:57.648504 7974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "db4de649d3d842bebb9e46ce80733ac3"
format_stamp: "Formatted at 2026-08-09 12:03:57 on dist-test-slave-sb2z"
server_key: "e24a86bab80e88903fa00636c525a0ba"
server_key_iv: "e35eaa37caa10d84d7d935f9c182a851"
server_key_version: "encryptionkey@0"
I20260809 12:03:57.653532 7974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:03:57.656910 7990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:57.658248 7974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 12:03:57.658443 7974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "db4de649d3d842bebb9e46ce80733ac3"
format_stamp: "Formatted at 2026-08-09 12:03:57 on dist-test-slave-sb2z"
server_key: "e24a86bab80e88903fa00636c525a0ba"
server_key_iv: "e35eaa37caa10d84d7d935f9c182a851"
server_key_version: "encryptionkey@0"
I20260809 12:03:57.658675 7974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:57.677517 7974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:57.678341 7974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:57.678664 7974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:57.697734 7974 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:44025
I20260809 12:03:57.697747 8067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:44025 every 8 connection(s)
I20260809 12:03:57.699028 7974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:57.702750 8068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:57.704430 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 7974
I20260809 12:03:57.704674 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:57.705498 589 external_mini_cluster.cc:1428] Setting key c860ac909224a2ba158a2c1cef0f8a90
I20260809 12:03:57.717022 8068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3: Bootstrap starting.
I20260809 12:03:57.719050 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:57.707447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.0.147.126:44025)
0809 12:03:57.708494 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:57.708504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:57.708544 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:03:57.710143 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:57.710168 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:57.710602 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:57.710899 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:57.711512 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:57.712872 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:57.714779 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:57.715412 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:57.715624 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:57.716822 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:57.716829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:57.716841 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:57.717385 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:57.717457 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:57.717482 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:57.717576 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:57.717790 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 12:03:57.718112 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":384,"threads_started":1}
I20260809 12:03:57.720374 8068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:57.721487 8068 log.cc:824] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:57.722898 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:03:57.724663 8068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3: No bootstrap required, opened a new log
I20260809 12:03:57.732328 8068 raft_consensus.cc:348] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4de649d3d842bebb9e46ce80733ac3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44025 } }
I20260809 12:03:57.732753 8068 raft_consensus.cc:374] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:57.732852 8068 raft_consensus.cc:729] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db4de649d3d842bebb9e46ce80733ac3, State: Initialized, Role: FOLLOWER
I20260809 12:03:57.733409 8068 consensus_queue.cc:260] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4de649d3d842bebb9e46ce80733ac3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44025 } }
I20260809 12:03:57.733646 8068 raft_consensus.cc:388] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:57.733773 8068 raft_consensus.cc:482] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:57.733951 8068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:57.736202 8068 raft_consensus.cc:504] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4de649d3d842bebb9e46ce80733ac3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44025 } }
I20260809 12:03:57.736713 8068 leader_election.cc:302] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db4de649d3d842bebb9e46ce80733ac3; no voters:
I20260809 12:03:57.737345 8068 leader_election.cc:288] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:57.737550 8074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:57.738346 8074 raft_consensus.cc:686] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [term 1 LEADER]: Becoming Leader. State: Replica: db4de649d3d842bebb9e46ce80733ac3, State: Running, Role: LEADER
I20260809 12:03:57.738933 8074 consensus_queue.cc:237] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4de649d3d842bebb9e46ce80733ac3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44025 } }
I20260809 12:03:57.739450 8068 sys_catalog.cc:564] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:57.742406 8075 sys_catalog.cc:455] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db4de649d3d842bebb9e46ce80733ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4de649d3d842bebb9e46ce80733ac3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44025 } } }
I20260809 12:03:57.742471 8077 sys_catalog.cc:455] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db4de649d3d842bebb9e46ce80733ac3. Latest consensus state: current_term: 1 leader_uuid: "db4de649d3d842bebb9e46ce80733ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4de649d3d842bebb9e46ce80733ac3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44025 } } }
I20260809 12:03:57.742893 8075 sys_catalog.cc:458] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:57.742902 8077 sys_catalog.cc:458] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:57.745922 8084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:57.750631 8084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:57.759064 8084 catalog_manager.cc:1347] Generated new cluster ID: 685e20442ffb446ca552f2794b4bafe4
I20260809 12:03:57.759210 8084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:57.773744 8084 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:57.775606 8084 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:57.791474 8084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db4de649d3d842bebb9e46ce80733ac3: Generated new TSK 0
I20260809 12:03:57.792503 8084 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 12:03:57.838434 8110 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 12:03:57.838672 8110 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 12:03:57.839849 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:57.817330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.0.147.126:44025)
0809 12:03:57.817540 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:57.817548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:57.818116 (+ 568us) server_negotiation.cc:407] Connection header received
0809 12:03:57.818423 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:57.818434 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:57.818522 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:57.818690 (+ 168us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:03:57.819249 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:57.820412 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:57.822065 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:57.822769 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:57.822893 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:57.823409 (+ 516us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:03:57.839146 (+ 15737us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 12:03:57.839503 (+ 357us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 12:03:57.839972 8071 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 12:03:57.841996 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:03:57.817267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:44025 (local address 127.0.0.1:55086)
0809 12:03:57.818014 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:57.818035 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:57.818175 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:57.818728 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:57.818732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:57.818742 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:03:57.819087 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:57.819095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:57.820620 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:57.820633 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:57.821915 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:57.821927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:57.822936 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:57.822940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:57.823096 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:57.823154 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:03:57.839439 (+ 16285us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 12:03:57.839858 (+ 419us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.147.126:44025: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":419,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.147.126:44025: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 12:03:57.844007 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 7974
I20260809 12:03:57.845155 8063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:57.940096 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 7974
2026-08-09T12:03:57Z chronyd exiting
I20260809 12:03:58.001101 589 test_util.cc:175] -----------------------------------------------
I20260809 12:03:58.001283 589 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786276990256303-589-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (777 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 12:03:58.005193 589 mini_oidc.cc:152] Starting JWKS server
I20260809 12:03:58.007135 589 webserver.cc:460] Webserver started at https://127.0.0.1:45329/ using document root <none> and password file <none>
I20260809 12:03:58.007390 589 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 12:03:58.008112 589 webserver.cc:460] Webserver started at http://127.0.0.1:45369/ using document root <none> and password file <none>
2026-08-09T12:03:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:58Z Disabled control of system clock
I20260809 12:03:58.044535 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:42241
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:34307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:42241
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45329/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/testchainca.cert with env {}
W20260809 12:03:58.374063 8123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:58.374419 8123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:58.374585 8123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:58.381929 8123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:58.382081 8123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:58.382174 8123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:58.382283 8123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:58.382371 8123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:03:58.382467 8123 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:03:58.382546 8123 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45329/jwks.json
I20260809 12:03:58.392402 8123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:34307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:42241
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:42241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45329/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:58.393663 8123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:58.394966 8123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:58.406502 8130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:58.406666 8129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:58.406807 8132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:58.407802 8123 server_base.cc:1034] running on GCE node
I20260809 12:03:58.408592 8123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:58.409925 8123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:58.411161 8123 hybrid_clock.cc:648] HybridClock initialized: now 1786277038411130 us; error 46 us; skew 500 ppm
I20260809 12:03:58.411501 8123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:03:58.413751 8123 webserver.cc:460] Webserver started at http://127.0.147.126:41437/ using document root <none> and password file <none>
I20260809 12:03:58.414572 8123 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:58.414707 8123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:58.415050 8123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:58.417879 8123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "7ddf067e49f1482fa950dbf9fca042ee"
format_stamp: "Formatted at 2026-08-09 12:03:58 on dist-test-slave-sb2z"
server_key: "fe31b1b178cf7bd444882af4a5602923"
server_key_iv: "0e89c0e585e4b2fdd24983c637a0e680"
server_key_version: "encryptionkey@0"
I20260809 12:03:58.418663 8123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "7ddf067e49f1482fa950dbf9fca042ee"
format_stamp: "Formatted at 2026-08-09 12:03:58 on dist-test-slave-sb2z"
server_key: "fe31b1b178cf7bd444882af4a5602923"
server_key_iv: "0e89c0e585e4b2fdd24983c637a0e680"
server_key_version: "encryptionkey@0"
I20260809 12:03:58.423655 8123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 12:03:58.427033 8139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:58.428388 8123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:58.428586 8123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "7ddf067e49f1482fa950dbf9fca042ee"
format_stamp: "Formatted at 2026-08-09 12:03:58 on dist-test-slave-sb2z"
server_key: "fe31b1b178cf7bd444882af4a5602923"
server_key_iv: "0e89c0e585e4b2fdd24983c637a0e680"
server_key_version: "encryptionkey@0"
I20260809 12:03:58.428803 8123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:58.439952 8123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:58.440719 8123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:58.441046 8123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:58.461130 8123 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:42241
I20260809 12:03:58.461156 8214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:42241 every 8 connection(s)
I20260809 12:03:58.462545 8123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:58.465574 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 8123
I20260809 12:03:58.465785 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:58.466632 589 external_mini_cluster.cc:1428] Setting key d41b9b9b52e551fe6ea200de8f4a0309
I20260809 12:03:58.467096 8215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:58.480670 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:58.468677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59666 (local address 127.0.147.126:42241)
0809 12:03:58.469679 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:58.469689 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:58.469734 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:03:58.471863 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:58.471890 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:58.472348 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:58.472685 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:58.473271 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:58.474615 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:58.476440 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:58.477079 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:58.477213 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:58.478461 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:58.478468 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:58.478480 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:58.479037 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:58.479117 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:58.479140 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:58.479232 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:58.479459 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 12:03:58.479760 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":369,"threads_started":1}
I20260809 12:03:58.481262 8215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee: Bootstrap starting.
I20260809 12:03:58.484511 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:03:58.484858 8215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:58.485836 8215 log.cc:824] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:58.488790 8215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee: No bootstrap required, opened a new log
I20260809 12:03:58.496256 8215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ddf067e49f1482fa950dbf9fca042ee" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42241 } }
I20260809 12:03:58.496605 8215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:58.496729 8215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ddf067e49f1482fa950dbf9fca042ee, State: Initialized, Role: FOLLOWER
I20260809 12:03:58.497328 8215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ddf067e49f1482fa950dbf9fca042ee" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42241 } }
I20260809 12:03:58.497557 8215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:58.497706 8215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:58.497875 8215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:58.500095 8215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ddf067e49f1482fa950dbf9fca042ee" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42241 } }
I20260809 12:03:58.500650 8215 leader_election.cc:302] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ddf067e49f1482fa950dbf9fca042ee; no voters:
I20260809 12:03:58.501382 8215 leader_election.cc:288] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:58.501499 8220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:58.502247 8220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [term 1 LEADER]: Becoming Leader. State: Replica: 7ddf067e49f1482fa950dbf9fca042ee, State: Running, Role: LEADER
I20260809 12:03:58.502799 8220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ddf067e49f1482fa950dbf9fca042ee" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42241 } }
I20260809 12:03:58.503803 8215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:58.505980 8222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ddf067e49f1482fa950dbf9fca042ee. Latest consensus state: current_term: 1 leader_uuid: "7ddf067e49f1482fa950dbf9fca042ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ddf067e49f1482fa950dbf9fca042ee" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42241 } } }
I20260809 12:03:58.506433 8222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:58.507007 8221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ddf067e49f1482fa950dbf9fca042ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ddf067e49f1482fa950dbf9fca042ee" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 42241 } } }
I20260809 12:03:58.507295 8221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:58.512848 8233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:58.518615 8233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:58.527508 8233 catalog_manager.cc:1347] Generated new cluster ID: c5cc6bcc30dc4472a23d51a1b55a05bb
I20260809 12:03:58.527701 8233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:58.549371 8233 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:58.551265 8233 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:58.566357 8233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ddf067e49f1482fa950dbf9fca042ee: Generated new TSK 0
I20260809 12:03:58.567446 8233 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 12:03:58.601732 8265 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 12:03:58.601863 8265 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 12:03:58.602929 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:58.583341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59680 (local address 127.0.147.126:42241)
0809 12:03:58.583553 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:58.583559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:58.583957 (+ 398us) server_negotiation.cc:407] Connection header received
0809 12:03:58.584155 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:58.584161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:58.584258 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:58.584415 (+ 157us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:03:58.584860 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:58.585967 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:58.587312 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:58.588081 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:58.588209 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:58.588683 (+ 474us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:03:58.602373 (+ 13690us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 12:03:58.602691 (+ 318us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59680: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 12:03:58.603089 8218 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59680: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 12:03:58.603327 8264 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:03:58.583211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:42241 (local address 127.0.0.1:59680)
0809 12:03:58.583854 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:58.583870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:58.583984 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:58.584440 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:58.584444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:58.584453 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:03:58.584720 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:58.584728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:58.586103 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:58.586106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:58.587176 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:58.587186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:58.588231 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:58.588235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:58.588389 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:58.588419 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:03:58.602621 (+ 14202us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 12:03:58.602907 (+ 286us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.147.126:42241: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":363,"threads_started":1}
I20260809 12:03:58.604485 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 8123
I20260809 12:03:58.605798 8205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:03:58.699519 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 8123
2026-08-09T12:03:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (755 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 12:03:58.760684 589 mini_oidc.cc:152] Starting JWKS server
I20260809 12:03:58.762589 589 webserver.cc:460] Webserver started at https://127.0.0.1:39419/ using document root <none> and password file <none>
I20260809 12:03:58.762814 589 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 12:03:58.763409 589 webserver.cc:460] Webserver started at http://127.0.0.1:36609/ using document root <none> and password file <none>
2026-08-09T12:03:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:58Z Disabled control of system clock
I20260809 12:03:58.800120 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:35145
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:46261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:35145
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39419/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 12:03:59.129287 8274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:59.129616 8274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:59.129782 8274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:59.136864 8274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:59.137019 8274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:59.137101 8274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:59.137212 8274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:59.137300 8274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:03:59.137392 8274 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:03:59.137475 8274 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39419/jwks.json
I20260809 12:03:59.146428 8274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:46261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:35145
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:35145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39419/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:59.147692 8274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:59.149467 8274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:59.161139 8280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:59.161172 8283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:59.161224 8281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:59.162362 8274 server_base.cc:1034] running on GCE node
I20260809 12:03:59.163199 8274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:59.164615 8274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:59.165856 8274 hybrid_clock.cc:648] HybridClock initialized: now 1786277039165815 us; error 40 us; skew 500 ppm
I20260809 12:03:59.166239 8274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:03:59.171952 8274 webserver.cc:460] Webserver started at http://127.0.147.126:35207/ using document root <none> and password file <none>
I20260809 12:03:59.172639 8274 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:59.172756 8274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:59.173122 8274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:59.176102 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "9be889e59a6a4a868f4c6712066b72c7"
format_stamp: "Formatted at 2026-08-09 12:03:59 on dist-test-slave-sb2z"
server_key: "71b4b29c46e6955ffd90bd2385718844"
server_key_iv: "fdb69abcc26b7a5fd1d1bb73ffa3c3a5"
server_key_version: "encryptionkey@0"
I20260809 12:03:59.176966 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "9be889e59a6a4a868f4c6712066b72c7"
format_stamp: "Formatted at 2026-08-09 12:03:59 on dist-test-slave-sb2z"
server_key: "71b4b29c46e6955ffd90bd2385718844"
server_key_iv: "fdb69abcc26b7a5fd1d1bb73ffa3c3a5"
server_key_version: "encryptionkey@0"
I20260809 12:03:59.182180 8274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:03:59.185571 8292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:59.186873 8274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:03:59.187076 8274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "9be889e59a6a4a868f4c6712066b72c7"
format_stamp: "Formatted at 2026-08-09 12:03:59 on dist-test-slave-sb2z"
server_key: "71b4b29c46e6955ffd90bd2385718844"
server_key_iv: "fdb69abcc26b7a5fd1d1bb73ffa3c3a5"
server_key_version: "encryptionkey@0"
I20260809 12:03:59.187294 8274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:59.210059 8274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:59.210934 8274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:59.211261 8274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:59.230285 8274 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:35145
I20260809 12:03:59.230305 8362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:35145 every 8 connection(s)
I20260809 12:03:59.231762 8274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:03:59.235414 8363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:03:59.239961 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 8274
I20260809 12:03:59.240195 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:03:59.241034 589 external_mini_cluster.cc:1428] Setting key 5b9e98b66cccbf75d7ba9709af5ba26e
I20260809 12:03:59.246979 8363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7: Bootstrap starting.
I20260809 12:03:59.250166 8363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7: Neither blocks nor log segments found. Creating new log.
I20260809 12:03:59.251230 8363 log.cc:824] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:03:59.254317 8363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7: No bootstrap required, opened a new log
I20260809 12:03:59.254606 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:59.242991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.0.147.126:35145)
0809 12:03:59.244029 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:59.244044 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:59.244087 (+ 43us) server_negotiation.cc:407] Connection header received
0809 12:03:59.245829 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:59.245853 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:59.246290 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:59.246678 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:03:59.247227 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:59.248595 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:59.249956 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:59.250711 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:59.250924 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:59.252182 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:03:59.252195 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:03:59.252213 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:03:59.252907 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:03:59.252983 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:03:59.253003 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:03:59.253089 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 12:03:59.253343 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 12:03:59.253684 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":407,"threads_started":1}
I20260809 12:03:59.258458 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:03:59.262629 8363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be889e59a6a4a868f4c6712066b72c7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35145 } }
I20260809 12:03:59.262971 8363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:03:59.263078 8363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be889e59a6a4a868f4c6712066b72c7, State: Initialized, Role: FOLLOWER
I20260809 12:03:59.263640 8363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be889e59a6a4a868f4c6712066b72c7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35145 } }
I20260809 12:03:59.263883 8363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:03:59.264022 8363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:03:59.264201 8363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:03:59.266556 8363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be889e59a6a4a868f4c6712066b72c7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35145 } }
I20260809 12:03:59.267038 8363 leader_election.cc:302] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9be889e59a6a4a868f4c6712066b72c7; no voters:
I20260809 12:03:59.267654 8363 leader_election.cc:288] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:03:59.267778 8371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:03:59.268517 8371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [term 1 LEADER]: Becoming Leader. State: Replica: 9be889e59a6a4a868f4c6712066b72c7, State: Running, Role: LEADER
I20260809 12:03:59.269074 8371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be889e59a6a4a868f4c6712066b72c7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35145 } }
I20260809 12:03:59.269872 8363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:03:59.272275 8375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9be889e59a6a4a868f4c6712066b72c7. Latest consensus state: current_term: 1 leader_uuid: "9be889e59a6a4a868f4c6712066b72c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be889e59a6a4a868f4c6712066b72c7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35145 } } }
I20260809 12:03:59.272723 8375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:59.273298 8372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9be889e59a6a4a868f4c6712066b72c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be889e59a6a4a868f4c6712066b72c7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35145 } } }
I20260809 12:03:59.273833 8372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:03:59.275179 8383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:03:59.280035 8383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:03:59.289412 8383 catalog_manager.cc:1347] Generated new cluster ID: 7d5b36242616411ab314c8cf4bd70b2e
I20260809 12:03:59.289532 8383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:03:59.310613 8383 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:03:59.311926 8383 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:03:59.320876 8383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9be889e59a6a4a868f4c6712066b72c7: Generated new TSK 0
I20260809 12:03:59.321916 8383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:03:59.350765 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:03:59.343237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.0.147.126:35145)
0809 12:03:59.343431 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:03:59.343441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:03:59.343851 (+ 410us) server_negotiation.cc:407] Connection header received
0809 12:03:59.344098 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:03:59.344105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:03:59.344167 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:03:59.344345 (+ 178us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:03:59.344884 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:59.345600 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:59.347021 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:03:59.347492 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:03:59.347664 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:59.348183 (+ 519us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:03:59.350102 (+ 1919us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39419/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 12:03:59.350444 (+ 342us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32866: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39419/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 12:03:59.350999 8412 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:03:59.343156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35145 (local address 127.0.0.1:32866)
0809 12:03:59.343755 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0809 12:03:59.343769 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:03:59.343918 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:03:59.344369 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:03:59.344375 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:03:59.344388 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:03:59.344734 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:59.344743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:59.345751 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:59.345759 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:59.346864 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:03:59.346874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:03:59.347744 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:03:59.347752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:03:59.347890 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:03:59.347922 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:03:59.350305 (+ 2383us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39419/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 12:03:59.350551 (+ 246us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.147.126:35145: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39419/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":325,"threads_started":1}
W20260809 12:03:59.351254 8368 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32866: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39419/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260809 12:03:59.354317 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 8274
I20260809 12:03:59.355458 8353 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 12:03:59.368614 8413 webserver.cc:533] Webserver: sslize failed:
E20260809 12:03:59.368762 8413 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 12:03:59.451210 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 8274
2026-08-09T12:03:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (757 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T12:03:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:03:59Z Disabled control of system clock
I20260809 12:03:59.556089 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:36897
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:34147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:36897
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/testchainca.cert with env {}
W20260809 12:03:59.884505 8418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:03:59.884847 8418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:03:59.884999 8418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:03:59.892133 8418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:03:59.892267 8418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:03:59.892344 8418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:03:59.892462 8418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:03:59.892539 8418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:03:59.892627 8418 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:03:59.892702 8418 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 12:03:59.902957 8418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:34147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:36897
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:36897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:03:59.904323 8418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:03:59.905607 8418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:03:59.917613 8423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:59.917697 8426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:03:59.917660 8424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:03:59.918591 8418 server_base.cc:1034] running on GCE node
I20260809 12:03:59.919358 8418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:03:59.920781 8418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:03:59.921980 8418 hybrid_clock.cc:648] HybridClock initialized: now 1786277039921945 us; error 27 us; skew 500 ppm
I20260809 12:03:59.922330 8418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:03:59.924606 8418 webserver.cc:460] Webserver started at http://127.0.147.126:45009/ using document root <none> and password file <none>
I20260809 12:03:59.925266 8418 fs_manager.cc:359] Metadata directory not provided
I20260809 12:03:59.925387 8418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:03:59.925735 8418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:03:59.928650 8418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb"
format_stamp: "Formatted at 2026-08-09 12:03:59 on dist-test-slave-sb2z"
server_key: "b337893ad0d6a49428ca8e3ddc6772e0"
server_key_iv: "3ace0e25aed8e88394d9eda6695203e3"
server_key_version: "encryptionkey@0"
I20260809 12:03:59.929445 8418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb"
format_stamp: "Formatted at 2026-08-09 12:03:59 on dist-test-slave-sb2z"
server_key: "b337893ad0d6a49428ca8e3ddc6772e0"
server_key_iv: "3ace0e25aed8e88394d9eda6695203e3"
server_key_version: "encryptionkey@0"
I20260809 12:03:59.934621 8418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:03:59.938547 8432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:03:59.939903 8418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:03:59.940093 8418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb"
format_stamp: "Formatted at 2026-08-09 12:03:59 on dist-test-slave-sb2z"
server_key: "b337893ad0d6a49428ca8e3ddc6772e0"
server_key_iv: "3ace0e25aed8e88394d9eda6695203e3"
server_key_version: "encryptionkey@0"
I20260809 12:03:59.940317 8418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:03:59.976630 8418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:03:59.977483 8418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:03:59.977787 8418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:03:59.996620 8418 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:36897
I20260809 12:03:59.996665 8497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:36897 every 8 connection(s)
I20260809 12:03:59.997972 8418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:04:00.001770 8499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:00.007335 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 8418
I20260809 12:04:00.007612 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:04:00.008277 589 external_mini_cluster.cc:1428] Setting key 991da310fafc8ebe02e0a417f64d58ca
I20260809 12:04:00.012578 8499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb: Bootstrap starting.
I20260809 12:04:00.016105 8499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:00.017369 8499 log.cc:824] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:00.021226 8499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb: No bootstrap required, opened a new log
I20260809 12:04:00.023200 8502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:00.010293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.0.147.126:36897)
0809 12:04:00.011353 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:00.011362 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:00.011402 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:04:00.013157 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:00.013180 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:00.013630 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:00.013935 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:00.014640 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:00.015937 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:00.017774 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:00.018917 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:00.019098 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:00.020463 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:00.020478 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:00.020534 (+ 56us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:00.021464 (+ 930us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:00.021553 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:00.021580 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:00.021677 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:00.021934 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 12:04:00.022269 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":443,"threads_started":1}
I20260809 12:04:00.027261 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:04:00.031268 8499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36897 } }
I20260809 12:04:00.031733 8499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:00.031853 8499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 548deffaf63f43dd8b7a3b43b4d5d5cb, State: Initialized, Role: FOLLOWER
I20260809 12:04:00.032474 8499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36897 } }
I20260809 12:04:00.032692 8499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:04:00.032842 8499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:04:00.033016 8499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:00.035151 8499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36897 } }
I20260809 12:04:00.035708 8499 leader_election.cc:302] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 548deffaf63f43dd8b7a3b43b4d5d5cb; no voters:
I20260809 12:04:00.036295 8499 leader_election.cc:288] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:04:00.036444 8504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:00.037894 8504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [term 1 LEADER]: Becoming Leader. State: Replica: 548deffaf63f43dd8b7a3b43b4d5d5cb, State: Running, Role: LEADER
I20260809 12:04:00.038501 8499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:00.038434 8504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36897 } }
I20260809 12:04:00.042738 8506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 548deffaf63f43dd8b7a3b43b4d5d5cb. Latest consensus state: current_term: 1 leader_uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36897 } } }
I20260809 12:04:00.042781 8505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548deffaf63f43dd8b7a3b43b4d5d5cb" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36897 } } }
I20260809 12:04:00.043171 8506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:00.043227 8505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:00.046118 8513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:00.050967 8513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:00.059346 8513 catalog_manager.cc:1347] Generated new cluster ID: 0ceae3fad9d94b0abf433cb5a5ffda81
I20260809 12:04:00.059463 8513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:00.075703 8513 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:04:00.077030 8513 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:00.086900 8513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 548deffaf63f43dd8b7a3b43b4d5d5cb: Generated new TSK 0
I20260809 12:04:00.087757 8513 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:04:00.164449 8536 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:00.102806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:36897 (local address 127.0.0.1:49736)
0809 12:04:00.103419 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:00.103435 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:00.103625 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:00.104093 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:00.104096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:00.104106 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:04:00.104336 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:00.104344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:00.105786 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:00.105789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:00.106864 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:00.106874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:00.107928 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:00.107933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:00.108060 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:00.108099 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:04:00.163855 (+ 55756us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 12:04:00.164121 (+ 266us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.147.126:36897: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":320,"threads_started":1}
I20260809 12:04:00.164431 8502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:00.102950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49736 (local address 127.0.147.126:36897)
0809 12:04:00.103156 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:00.103164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:00.103546 (+ 382us) server_negotiation.cc:407] Connection header received
0809 12:04:00.103806 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:00.103812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:00.103913 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:00.104086 (+ 173us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:04:00.104486 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:00.105647 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:00.107005 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:00.107749 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:00.107879 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:00.108367 (+ 488us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:04:00.163521 (+ 55154us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 12:04:00.164053 (+ 532us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49736: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 12:04:00.164618 8502 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49736: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260809 12:04:00.165663 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 8418
I20260809 12:04:00.166965 8491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:00.264768 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 8418
2026-08-09T12:04:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (788 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (428 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (428 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:04:01 dist-test-slave-sb2z krb5kdc[8566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:04:01 dist-test-slave-sb2z krb5kdc[8566](info): set up 2 sockets
Aug 09 12:04:01 dist-test-slave-sb2z krb5kdc[8566](info): commencing operation
krb5kdc: starting...
W20260809 12:04:02.410583 589 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:04:02 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:04:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:04:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:04:02.663084 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:46633
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:46633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:03.006990 8580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:03.007266 8580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:03.007450 8580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:03.014606 8580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:03.014727 8580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:03.014832 8580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:03.014948 8580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:03.015034 8580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:03.025171 8580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:46633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:46633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:03.026571 8580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:03.027876 8580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:03.039361 8585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:03.039412 8588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:03.039454 8586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:03.040345 8580 server_base.cc:1034] running on GCE node
I20260809 12:04:03.041225 8580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:03.042568 8580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:03.043778 8580 hybrid_clock.cc:648] HybridClock initialized: now 1786277043043732 us; error 31 us; skew 500 ppm
I20260809 12:04:03.044116 8580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:03 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277043, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:03.047163 8580 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:04:03.048620 8580 webserver.cc:460] Webserver started at http://127.0.147.126:34357/ using document root <none> and password file <none>
I20260809 12:04:03.049664 8580 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:03.049798 8580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:03.050112 8580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:03.053439 8580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "56068cb0100648829d664f1e78ab1d1d"
format_stamp: "Formatted at 2026-08-09 12:04:03 on dist-test-slave-sb2z"
server_key: "accece98ff9f0ad661031251855b64c2"
server_key_iv: "8620d13e4c7dfa4db9d7db275ee78272"
server_key_version: "encryptionkey@0"
I20260809 12:04:03.054266 8580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "56068cb0100648829d664f1e78ab1d1d"
format_stamp: "Formatted at 2026-08-09 12:04:03 on dist-test-slave-sb2z"
server_key: "accece98ff9f0ad661031251855b64c2"
server_key_iv: "8620d13e4c7dfa4db9d7db275ee78272"
server_key_version: "encryptionkey@0"
I20260809 12:04:03.059861 8580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:04:03.063742 8601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:03.065133 8580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:04:03.065313 8580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "56068cb0100648829d664f1e78ab1d1d"
format_stamp: "Formatted at 2026-08-09 12:04:03 on dist-test-slave-sb2z"
server_key: "accece98ff9f0ad661031251855b64c2"
server_key_iv: "8620d13e4c7dfa4db9d7db275ee78272"
server_key_version: "encryptionkey@0"
I20260809 12:04:03.065495 8580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:03.085404 8580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:03.089546 8580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:03.089836 8580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:03.108935 8580 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:46633
I20260809 12:04:03.108969 8665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:46633 every 8 connection(s)
I20260809 12:04:03.110320 8580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:04:03.113965 8667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:03.117951 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 8580
I20260809 12:04:03.118182 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:04:03.119005 589 external_mini_cluster.cc:1428] Setting key 86e4e4b2d5b520fc4b29387baf714ee8
I20260809 12:04:03.126039 8667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d: Bootstrap starting.
I20260809 12:04:03.129633 8667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:03.130756 8667 log.cc:824] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:03.134086 8667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d: No bootstrap required, opened a new log
Aug 09 12:04:03 dist-test-slave-sb2z krb5kdc[8566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:03.141776 8667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56068cb0100648829d664f1e78ab1d1d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46633 } }
I20260809 12:04:03.142177 8667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:03.142279 8667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56068cb0100648829d664f1e78ab1d1d, State: Initialized, Role: FOLLOWER
I20260809 12:04:03.142864 8667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56068cb0100648829d664f1e78ab1d1d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46633 } }
I20260809 12:04:03.143121 8667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:04:03.143275 8667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:04:03.143450 8667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:03.143530 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:03.121159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.0.147.126:46633)
0809 12:04:03.121930 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:03.121941 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:03.121981 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:04:03.124574 (+ 2593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:03.124596 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:03.124952 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:03.125270 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:03.127047 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:03.128429 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:03.130228 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:03.130922 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:03.131066 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:03.135816 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:03.135823 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:03.135839 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:03.135878 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:03.139024 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:03.139783 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:03.139795 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:03.139800 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:03.139972 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:03.140393 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:03.140398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:03.140401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:03.140947 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:03.141283 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:03.141720 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 12:04:03.142175 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":369,"threads_started":1}
I20260809 12:04:03.145778 8667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56068cb0100648829d664f1e78ab1d1d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46633 } }
I20260809 12:04:03.146301 8667 leader_election.cc:302] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56068cb0100648829d664f1e78ab1d1d; no voters:
I20260809 12:04:03.146890 8667 leader_election.cc:288] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:04:03.147044 8674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:03.148372 8674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [term 1 LEADER]: Becoming Leader. State: Replica: 56068cb0100648829d664f1e78ab1d1d, State: Running, Role: LEADER
I20260809 12:04:03.148893 8674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56068cb0100648829d664f1e78ab1d1d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46633 } }
I20260809 12:04:03.149101 8667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:03.151870 8676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56068cb0100648829d664f1e78ab1d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56068cb0100648829d664f1e78ab1d1d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46633 } } }
I20260809 12:04:03.151962 8677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56068cb0100648829d664f1e78ab1d1d. Latest consensus state: current_term: 1 leader_uuid: "56068cb0100648829d664f1e78ab1d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56068cb0100648829d664f1e78ab1d1d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46633 } } }
I20260809 12:04:03.152523 8677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:03.152457 8676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:03.159314 8686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:03.164860 8686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:03.174280 8686 catalog_manager.cc:1347] Generated new cluster ID: fd9181056e0c426a9b64006dddb2f77c
I20260809 12:04:03.174415 8686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:03.193224 8686 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:04:03.194555 8686 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:03.217024 8686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56068cb0100648829d664f1e78ab1d1d: Generated new TSK 0
I20260809 12:04:03.218120 8686 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:04:03.288528 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46633
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:03.629472 8704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:03.629740 8704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:03.629834 8704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:03.637151 8704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:03.637261 8704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:03.637374 8704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:04:03.647331 8704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:03.648584 8704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:03.649952 8704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:03.663347 8720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:03.663355 8716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:03.663637 8715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:03.664180 8704 server_base.cc:1034] running on GCE node
I20260809 12:04:03.664788 8704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:03.665685 8704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:03.666917 8704 hybrid_clock.cc:648] HybridClock initialized: now 1786277043666842 us; error 85 us; skew 500 ppm
I20260809 12:04:03.667239 8704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:03 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277043, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:03.670753 8704 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:04:03.672194 8704 webserver.cc:460] Webserver started at http://127.0.147.65:33935/ using document root <none> and password file <none>
I20260809 12:04:03.672883 8704 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:03.673030 8704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:03.673338 8704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:03.676277 8704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "ce2462bda0aa48cb9db13e53903e571e"
format_stamp: "Formatted at 2026-08-09 12:04:03 on dist-test-slave-sb2z"
server_key: "a1940fe6f5b2dfa3f6bc7dd8b933e826"
server_key_iv: "6256a94880682ef5bcba8520ad216277"
server_key_version: "encryptionkey@0"
I20260809 12:04:03.677030 8704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "ce2462bda0aa48cb9db13e53903e571e"
format_stamp: "Formatted at 2026-08-09 12:04:03 on dist-test-slave-sb2z"
server_key: "a1940fe6f5b2dfa3f6bc7dd8b933e826"
server_key_iv: "6256a94880682ef5bcba8520ad216277"
server_key_version: "encryptionkey@0"
I20260809 12:04:03.681984 8704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:04:03.685339 8729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:03.686685 8704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:04:03.686867 8704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "ce2462bda0aa48cb9db13e53903e571e"
format_stamp: "Formatted at 2026-08-09 12:04:03 on dist-test-slave-sb2z"
server_key: "a1940fe6f5b2dfa3f6bc7dd8b933e826"
server_key_iv: "6256a94880682ef5bcba8520ad216277"
server_key_version: "encryptionkey@0"
I20260809 12:04:03.687078 8704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:03.699785 8704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:03.703462 8704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:03.703779 8704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:03.704674 8704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:03.705994 8704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:03.706118 8704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:03.706267 8704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:03.706347 8704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:03.740401 8704 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:33481
I20260809 12:04:03.740454 8880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:33481 every 8 connection(s)
I20260809 12:04:03.741724 8704 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:04:03.743458 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 8704
I20260809 12:04:03.743698 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:04:03.744550 589 external_mini_cluster.cc:1428] Setting key 8bbe25ccdf98f589dc9657f29319c20c
Aug 09 12:04:03 dist-test-slave-sb2z krb5kdc[8566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277043, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:03.764300 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:03.744619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:34003 (local address 127.0.147.126:46633)
0809 12:04:03.745157 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:03.745165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:03.746247 (+ 1082us) server_negotiation.cc:407] Connection header received
0809 12:04:03.748151 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:03.748160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:03.748258 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:03.748382 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:03.750018 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:03.751205 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:03.752791 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:03.753532 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:03.753686 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:03.757568 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:03.757574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:03.757577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:03.757610 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:03.760970 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:03.761700 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:03.761709 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:03.761716 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:03.761867 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:03.762282 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:03.762290 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:03.762296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:03.762685 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:03.762875 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:03.763522 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 12:04:03.763807 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":255,"threads_started":1}
I20260809 12:04:03.765033 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:03.745212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:46633 (local address 127.0.147.65:34003)
0809 12:04:03.746073 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:03.746124 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:03.747867 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:03.748577 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:03.748586 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:03.749197 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:03.749811 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:03.749852 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:03.751345 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:03.751349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:03.752643 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:03.752655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:03.753745 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:03.753749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:03.753945 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:03.754663 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:03.754684 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:03.757344 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:03.761124 (+ 3780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:03.761131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:03.761143 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:03.761563 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:03.761973 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:03.761976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:03.761978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:03.762162 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:03.762816 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:03.762821 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:03.763127 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 12:04:03.763344 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 12:04:03.763880 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":369,"threads_started":1}
I20260809 12:04:03.766655 8882 heartbeater.cc:347] Connected to a master server at 127.0.147.126:46633
I20260809 12:04:03.766975 8882 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:03.767776 8882 heartbeater.cc:511] Master 127.0.147.126:46633 requested a full tablet report, sending...
I20260809 12:04:03.769901 8622 ts_manager.cc:194] Registered new tserver with Master: ce2462bda0aa48cb9db13e53903e571e (127.0.147.65:33481)
I20260809 12:04:03.771311 8622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:34003
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:04:03.803980 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46633
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:04.144028 8889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:04.144290 8889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:04.144462 8889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:04.151575 8889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:04.151698 8889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:04.151873 8889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:04:04.161296 8889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:04.162576 8889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:04.163906 8889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:04.177178 8898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:04.177152 8900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:04.177305 8897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:04.178018 8889 server_base.cc:1034] running on GCE node
I20260809 12:04:04.178725 8889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:04.179702 8889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:04.180941 8889 hybrid_clock.cc:648] HybridClock initialized: now 1786277044180899 us; error 57 us; skew 500 ppm
I20260809 12:04:04.181269 8889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:04.188699 8889 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:04:04.190254 8889 webserver.cc:460] Webserver started at http://127.0.147.66:45723/ using document root <none> and password file <none>
I20260809 12:04:04.190945 8889 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:04.191104 8889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:04.191411 8889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:04.194339 8889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "9a17b327ff074c62823083678c5ac424"
format_stamp: "Formatted at 2026-08-09 12:04:04 on dist-test-slave-sb2z"
server_key: "7435632d16d2e2e8a8bf5d56e0482363"
server_key_iv: "123e64314bfbb30d9fb0fcc9f8eb0131"
server_key_version: "encryptionkey@0"
I20260809 12:04:04.195153 8889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "9a17b327ff074c62823083678c5ac424"
format_stamp: "Formatted at 2026-08-09 12:04:04 on dist-test-slave-sb2z"
server_key: "7435632d16d2e2e8a8bf5d56e0482363"
server_key_iv: "123e64314bfbb30d9fb0fcc9f8eb0131"
server_key_version: "encryptionkey@0"
I20260809 12:04:04.200382 8889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:04:04.203823 8911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:04.205137 8889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:04:04.205314 8889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "9a17b327ff074c62823083678c5ac424"
format_stamp: "Formatted at 2026-08-09 12:04:04 on dist-test-slave-sb2z"
server_key: "7435632d16d2e2e8a8bf5d56e0482363"
server_key_iv: "123e64314bfbb30d9fb0fcc9f8eb0131"
server_key_version: "encryptionkey@0"
I20260809 12:04:04.205523 8889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:04.239145 8889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:04.243336 8889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:04.243683 8889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:04.244635 8889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:04.245947 8889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:04.246019 8889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:04.246111 8889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:04.246158 8889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:04.280682 8889 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:42249
I20260809 12:04:04.280764 9081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:42249 every 8 connection(s)
I20260809 12:04:04.281975 8889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:04:04.289033 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 8889
I20260809 12:04:04.289229 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:04:04.289793 589 external_mini_cluster.cc:1428] Setting key 5e1f49073cf8c8c28295777cca620949
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:04.304775 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:04.284591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:51215 (local address 127.0.147.126:46633)
0809 12:04:04.285056 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:04.285064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:04.286093 (+ 1029us) server_negotiation.cc:407] Connection header received
0809 12:04:04.287909 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:04.287913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:04.287979 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:04.288075 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:04.290034 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.290748 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.292797 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.293215 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.293311 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:04.297722 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:04.297731 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:04.297739 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:04.297785 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:04.301613 (+ 3828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.302380 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.302388 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.302394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.302566 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.302990 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.302998 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.303004 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.303377 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:04.303564 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:04.304327 (+ 763us) server_negotiation.cc:299] Negotiation successful
0809 12:04:04.304492 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":209,"threads_started":1}
I20260809 12:04:04.306309 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:04.285164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:46633 (local address 127.0.147.66:51215)
0809 12:04:04.285926 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:04.285963 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:04.287656 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:04.288273 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:04.288281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:04.288986 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:04.289800 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:04.289827 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.290916 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.290923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:04.292650 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:04.292666 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.293446 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.293457 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:04.293900 (+ 443us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:04.294779 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:04.294819 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:04.297494 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:04.301760 (+ 4266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:04.301767 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:04.301778 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:04.302232 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:04.302682 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:04.302685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:04.302688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:04.302871 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:04.303492 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:04.303498 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:04.303894 (+ 396us) client_negotiation.cc:769] Sending connection context
0809 12:04:04.304231 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 12:04:04.304799 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":329,"threads_started":1}
I20260809 12:04:04.307749 9082 heartbeater.cc:347] Connected to a master server at 127.0.147.126:46633
I20260809 12:04:04.308071 9082 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:04.308876 9082 heartbeater.cc:511] Master 127.0.147.126:46633 requested a full tablet report, sending...
I20260809 12:04:04.310494 8622 ts_manager.cc:194] Registered new tserver with Master: 9a17b327ff074c62823083678c5ac424 (127.0.147.66:42249)
I20260809 12:04:04.311775 8622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:51215
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:04:04.350297 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46633
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:04.683506 9089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:04.683800 9089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:04.683897 9089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:04.690824 9089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:04.690976 9089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:04.691169 9089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:04:04.701489 9089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:04.702720 9089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:04.703989 9089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:04.717044 9102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:04.717186 9099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:04.717140 9097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:04.718024 9089 server_base.cc:1034] running on GCE node
I20260809 12:04:04.718684 9089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:04.719625 9089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:04.720832 9089 hybrid_clock.cc:648] HybridClock initialized: now 1786277044720791 us; error 34 us; skew 500 ppm
I20260809 12:04:04.721112 9089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:04.724218 9089 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:04:04.725528 9089 webserver.cc:460] Webserver started at http://127.0.147.67:36319/ using document root <none> and password file <none>
I20260809 12:04:04.726176 9089 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:04.726310 9089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:04.726624 9089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:04.729598 9089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "d5acde97cf034a6a86c3e715023dd81a"
format_stamp: "Formatted at 2026-08-09 12:04:04 on dist-test-slave-sb2z"
server_key: "15accfa482016a9f2652c6c3090808e5"
server_key_iv: "4158b6df38fea6571d6ea338f8dd6774"
server_key_version: "encryptionkey@0"
I20260809 12:04:04.730378 9089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "d5acde97cf034a6a86c3e715023dd81a"
format_stamp: "Formatted at 2026-08-09 12:04:04 on dist-test-slave-sb2z"
server_key: "15accfa482016a9f2652c6c3090808e5"
server_key_iv: "4158b6df38fea6571d6ea338f8dd6774"
server_key_version: "encryptionkey@0"
I20260809 12:04:04.735316 9089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:04:04.738600 9113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:04.740060 9089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:04:04.740232 9089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "d5acde97cf034a6a86c3e715023dd81a"
format_stamp: "Formatted at 2026-08-09 12:04:04 on dist-test-slave-sb2z"
server_key: "15accfa482016a9f2652c6c3090808e5"
server_key_iv: "4158b6df38fea6571d6ea338f8dd6774"
server_key_version: "encryptionkey@0"
I20260809 12:04:04.740438 9089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:04.760152 9089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:04.764531 9089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:04.764851 9089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:04.765815 9089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:04.767201 9089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:04.767309 9089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:04.767467 9089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:04.767582 9089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:04.774421 8882 heartbeater.cc:503] Master 127.0.147.126:46633 was elected leader, sending a full tablet report...
I20260809 12:04:04.802309 9089 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:37173
I20260809 12:04:04.802386 9271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:37173 every 8 connection(s)
I20260809 12:04:04.803642 9089 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:04:04.804816 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 9089
I20260809 12:04:04.804996 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:04:04.805621 589 external_mini_cluster.cc:1428] Setting key 3f86e58ea82b40b50c78ece9232222cf
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:04.823843 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:04.806532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:58995 (local address 127.0.147.126:46633)
0809 12:04:04.807289 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:04.807297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:04.808208 (+ 911us) server_negotiation.cc:407] Connection header received
0809 12:04:04.810154 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:04.810158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:04.810232 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:04.810324 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:04.811851 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.812599 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.814078 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.814459 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.814569 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:04.817828 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:04.817832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:04.817836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:04.817867 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:04.820935 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.821764 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.821768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.821770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.821886 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.822268 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.822272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.822273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.822563 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:04.822683 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:04.823317 (+ 634us) server_negotiation.cc:299] Negotiation successful
0809 12:04:04.823470 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":530,"threads_started":1}
I20260809 12:04:04.824662 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:04.807149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:46633 (local address 127.0.147.67:58995)
0809 12:04:04.808039 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:04.808093 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:04.809917 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:04.810568 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:04.810575 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:04.811067 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:04.811680 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:04.811700 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.812729 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.812733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:04.813912 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:04.813924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.814650 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.814655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:04.814839 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:04.815498 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:04.815589 (+ 91us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:04.817658 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:04.821071 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:04.821079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:04.821094 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:04.821645 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:04.821976 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:04.821979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:04.821982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:04.822166 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:04.822686 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:04.822692 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:04.822982 (+ 290us) client_negotiation.cc:769] Sending connection context
0809 12:04:04.823209 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 12:04:04.823629 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":385,"threads_started":1}
I20260809 12:04:04.825924 9273 heartbeater.cc:347] Connected to a master server at 127.0.147.126:46633
I20260809 12:04:04.826247 9273 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:04.826969 9273 heartbeater.cc:511] Master 127.0.147.126:46633 requested a full tablet report, sending...
I20260809 12:04:04.828513 8622 ts_manager.cc:194] Registered new tserver with Master: d5acde97cf034a6a86c3e715023dd81a (127.0.147.67:37173)
I20260809 12:04:04.829282 8622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:58995
I20260809 12:04:04.833449 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/test-admin.keytab.
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:04.869194 589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 12:04:04.874760 9284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:04.871977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57290 (local address 127.0.147.65:33481)
0809 12:04:04.872694 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:04.872701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:04.872726 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:04:04.872788 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:04.872794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:04.873012 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:04.873196 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:04.874262 (+ 1066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57290: BlockingRecv error: recv got EOF from 127.0.0.1:57290 (error 108)
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":316,"threads_started":1}
W20260809 12:04:04.874884 589 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5cc' of type FILE
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:04.876745 589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:04.878151 589 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:04:04.893054 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:04.879719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.0.147.65:33481)
0809 12:04:04.879945 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:04.879951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:04.880163 (+ 212us) server_negotiation.cc:407] Connection header received
0809 12:04:04.880300 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:04.880303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:04.880366 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:04.880457 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:04.881443 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.882372 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.883620 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.884168 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.884271 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:04.887726 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:04.887737 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:04.887750 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:04.887780 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:04.890350 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.891021 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.891029 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.891031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.891154 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.891513 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.891519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.891542 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.891888 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:04.892210 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:04.892534 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 12:04:04.892796 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 12:04:04.898499 9284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:04.896846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.0.147.65:33481)
0809 12:04:04.897004 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:04.897009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:04.897358 (+ 349us) server_negotiation.cc:407] Connection header received
0809 12:04:04.897465 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:04.897468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:04.897528 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:04.897602 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:04.898240 (+ 638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57318: BlockingRecv error: recv got EOF from 127.0.0.1:57318 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 12:04:04.898718 589 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5cc' of type FILE
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:04.900161 589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:04.901427 589 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:04:04.915256 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:04.902615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.0.147.65:33481)
0809 12:04:04.902828 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:04.902836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:04.903103 (+ 267us) server_negotiation.cc:407] Connection header received
0809 12:04:04.903278 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:04.903283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:04.903341 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:04.903435 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:04.904375 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.905051 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.906241 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.906656 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.906788 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:04.910336 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:04.910340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:04.910343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:04.910372 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:04.912828 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.913465 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.913472 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.913476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.913603 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.913966 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.913971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.913975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.914236 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:04.914498 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:04.914807 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 12:04:04.915001 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 12:04:04.919533 9284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:04.917679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57344 (local address 127.0.147.65:33481)
0809 12:04:04.917883 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:04.917888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:04.918141 (+ 253us) server_negotiation.cc:407] Connection header received
0809 12:04:04.918335 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:04.918340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:04.918410 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:04.918504 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:04.919313 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57344: BlockingRecv error: recv got EOF from 127.0.0.1:57344 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 12:04:04.919752 589 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786276990256303-589-0/krb5kdc/krb5cc' of type FILE
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:04.921234 589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:04.922465 589 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 12:04:04 dist-test-slave-sb2z krb5kdc[8566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:04:04.936442 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:04.923683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.0.147.65:33481)
0809 12:04:04.923860 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:04.923867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:04.924177 (+ 310us) server_negotiation.cc:407] Connection header received
0809 12:04:04.924357 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:04.924361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:04.924418 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:04.924523 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:04.925407 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.926184 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.927411 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.927973 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.928102 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:04.931429 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:04.931433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:04.931437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:04.931465 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:04.933849 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.934493 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.934499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.934502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.934642 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.935009 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.935015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.935021 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.935267 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:04.935626 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:04.935910 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 12:04:04.936143 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 12:04:04.937920 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 8704
I20260809 12:04:04.949610 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:04.938697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57354 (local address 127.0.147.65:33481)
0809 12:04:04.938911 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:04.938917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:04.939362 (+ 445us) server_negotiation.cc:407] Connection header received
0809 12:04:04.939479 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:04.939482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:04.939585 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:04.939668 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:04.940547 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.941183 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.942310 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:04.942698 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:04.942783 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:04.945041 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:04.945046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:04.945049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:04.945076 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:04.947355 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.947953 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.947957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.947959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.948076 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:04.948400 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:04.948403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:04.948405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:04.948630 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:04.948871 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:04.949147 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 12:04:04.949293 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:04:04.950197 8874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:05.036046 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 8704
I20260809 12:04:05.060729 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 8889
Aug 09 12:04:05 dist-test-slave-sb2z krb5kdc[8566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:04:05.076684 9303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:05.061654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.0.147.66:42249)
0809 12:04:05.062154 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:05.062161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:05.062186 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:04:05.062247 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:05.062253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:05.062453 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:05.062645 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:05.063768 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:05.064801 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:05.066063 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:05.066621 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:05.066764 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:05.071001 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:05.071012 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:05.071027 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:05.071066 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:05.073816 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:05.074504 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:05.074512 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:05.074514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:05.074648 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:05.075034 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:05.075039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:05.075042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:05.075387 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:05.075739 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:05.076065 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 12:04:05.076331 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":253,"threads_started":1}
I20260809 12:04:05.077996 9077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:05.162062 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 8889
I20260809 12:04:05.185945 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 9089
Aug 09 12:04:05 dist-test-slave-sb2z krb5kdc[8566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:04:05.202332 9305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:05.186963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57596 (local address 127.0.147.67:37173)
0809 12:04:05.187702 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:05.187709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:05.187740 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:04:05.187802 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:05.187808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:05.188019 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:05.188196 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:05.189447 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:05.190896 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:05.192283 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:05.192845 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:05.192946 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:05.196913 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:05.196924 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:05.196937 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:05.196976 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:05.199814 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:05.200439 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:05.200446 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:05.200448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:05.200571 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:05.200923 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:05.200926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:05.200928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:05.201254 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:05.201498 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:05.201734 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 12:04:05.202014 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":353,"threads_started":1}
I20260809 12:04:05.203692 9266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:05.286648 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 9089
I20260809 12:04:05.309326 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 8580
I20260809 12:04:05.310513 8660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:05.418797 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 8580
2026-08-09T12:04:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4303 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:04:05 dist-test-slave-sb2z krb5kdc[9313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:04:05 dist-test-slave-sb2z krb5kdc[9313](info): set up 2 sockets
Aug 09 12:04:05 dist-test-slave-sb2z krb5kdc[9313](info): commencing operation
krb5kdc: starting...
W20260809 12:04:06.734591 589 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.004s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:04:06 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:04:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:04:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.126.keytab.
Entry for principal oryx/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.126.keytab.
I20260809 12:04:06.988552 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:35267
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.126.keytab
--principal=oryx/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:35267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:07.336110 9327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:07.336398 9327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:07.336562 9327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:07.343885 9327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:07.344010 9327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:07.344125 9327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:07.344223 9327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:07.344287 9327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:07.354164 9327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:35267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.126.keytab
--principal=oryx/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:35267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:07.355629 9327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:07.356988 9327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:07.368868 9336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:07.368932 9334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:07.368974 9332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:07.369850 9327 server_base.cc:1034] running on GCE node
I20260809 12:04:07.370615 9327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:07.372051 9327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:07.373292 9327 hybrid_clock.cc:648] HybridClock initialized: now 1786277047373260 us; error 43 us; skew 500 ppm
I20260809 12:04:07.373637 9327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:07 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277047, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:07.376881 9327 init.cc:377] Logged in from keytab as oryx/127.0.147.126@KRBTEST.COM (short username oryx)
I20260809 12:04:07.378280 9327 webserver.cc:460] Webserver started at http://127.0.147.126:38639/ using document root <none> and password file <none>
I20260809 12:04:07.378942 9327 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:07.379096 9327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:07.379402 9327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:07.382844 9327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "7b99ca339e2e440ba634712c2a35f0bf"
format_stamp: "Formatted at 2026-08-09 12:04:07 on dist-test-slave-sb2z"
server_key: "62213a91d5229c4659d1b2e6de39f75b"
server_key_iv: "8564ba14994e35328d92c0dddf0feff8"
server_key_version: "encryptionkey@0"
I20260809 12:04:07.383687 9327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "7b99ca339e2e440ba634712c2a35f0bf"
format_stamp: "Formatted at 2026-08-09 12:04:07 on dist-test-slave-sb2z"
server_key: "62213a91d5229c4659d1b2e6de39f75b"
server_key_iv: "8564ba14994e35328d92c0dddf0feff8"
server_key_version: "encryptionkey@0"
I20260809 12:04:07.388788 9327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:04:07.392190 9352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:07.393473 9327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:04:07.393656 9327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "7b99ca339e2e440ba634712c2a35f0bf"
format_stamp: "Formatted at 2026-08-09 12:04:07 on dist-test-slave-sb2z"
server_key: "62213a91d5229c4659d1b2e6de39f75b"
server_key_iv: "8564ba14994e35328d92c0dddf0feff8"
server_key_version: "encryptionkey@0"
I20260809 12:04:07.393850 9327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:07.417773 9327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:07.422258 9327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:07.422552 9327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:07.441646 9327 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:35267
I20260809 12:04:07.441689 9419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:35267 every 8 connection(s)
I20260809 12:04:07.443049 9327 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:04:07.445479 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 9327
I20260809 12:04:07.445703 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:04:07.446383 589 external_mini_cluster.cc:1428] Setting key 480b10bbff08b66c73fb98ccf413dd71
I20260809 12:04:07.446935 9420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:07.461571 9420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: Bootstrap starting.
Aug 09 12:04:07 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:07.465729 9420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:07.467172 9420 log.cc:824] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:07.470862 9420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: No bootstrap required, opened a new log
I20260809 12:04:07.472172 9424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:07.448548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.0.147.126:35267)
0809 12:04:07.449381 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:07.449397 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:07.449450 (+ 53us) server_negotiation.cc:407] Connection header received
0809 12:04:07.451931 (+ 2481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:07.451961 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:07.452474 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:07.452935 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:07.454221 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:07.456297 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:07.457631 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:07.458760 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:07.458951 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:07.463112 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:07.463124 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:07.463152 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:07.463214 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:07.467378 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:07.468295 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:07.468301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:07.468308 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:07.468530 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:07.468958 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:07.468962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:07.468965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:07.469629 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:07.469959 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:07.470323 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 12:04:07.470773 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":341,"threads_started":1}
I20260809 12:04:07.479166 9420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } }
I20260809 12:04:07.479604 9420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:07.479727 9420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b99ca339e2e440ba634712c2a35f0bf, State: Initialized, Role: FOLLOWER
I20260809 12:04:07.480264 9420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } }
I20260809 12:04:07.480510 9420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:04:07.480664 9420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:04:07.480830 9420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:07.482991 9420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } }
I20260809 12:04:07.483477 9420 leader_election.cc:302] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b99ca339e2e440ba634712c2a35f0bf; no voters:
I20260809 12:04:07.484156 9420 leader_election.cc:288] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:04:07.484294 9427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:07.485059 9427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 1 LEADER]: Becoming Leader. State: Replica: 7b99ca339e2e440ba634712c2a35f0bf, State: Running, Role: LEADER
I20260809 12:04:07.485646 9427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } }
I20260809 12:04:07.486258 9420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:07.488459 9430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b99ca339e2e440ba634712c2a35f0bf. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } }
I20260809 12:04:07.488602 9428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } }
I20260809 12:04:07.488950 9430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:07.488955 9428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:07.492024 9436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:07.497287 9436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:07.506083 9436 catalog_manager.cc:1347] Generated new cluster ID: 6fe03249f99e4815a346c82328311b7a
I20260809 12:04:07.506203 9436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:07.550429 9436 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:04:07.552395 9436 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:07.560137 9436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: Generated new TSK 0
I20260809 12:04:07.561187 9436 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab.
Entry for principal oryx/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab.
I20260809 12:04:07.638971 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:35267
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab
--principal=oryx/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:07.976933 9455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:07.977192 9455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:07.977384 9455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:07.984707 9455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:07.984810 9455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:07.985013 9455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:04:07.996603 9455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab
--principal=oryx/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:35267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:07.998027 9455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:07.999421 9455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:08.012532 9462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:08.012532 9460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:08.012717 9464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:08.013484 9455 server_base.cc:1034] running on GCE node
I20260809 12:04:08.014130 9455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:08.014979 9455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:08.016211 9455 hybrid_clock.cc:648] HybridClock initialized: now 1786277048016166 us; error 58 us; skew 500 ppm
I20260809 12:04:08.016522 9455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:08 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277048, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:08.019707 9455 init.cc:377] Logged in from keytab as oryx/127.0.147.65@KRBTEST.COM (short username oryx)
I20260809 12:04:08.021108 9455 webserver.cc:460] Webserver started at http://127.0.147.65:46071/ using document root <none> and password file <none>
I20260809 12:04:08.021782 9455 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:08.021927 9455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:08.022235 9455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:08.025151 9455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "acc522b25d284580a2af45212bc42e40"
format_stamp: "Formatted at 2026-08-09 12:04:08 on dist-test-slave-sb2z"
server_key: "12ebb8ff9ba4f9dcdbac6e305ad82352"
server_key_iv: "e2596bc258c724262aee0b6bf6099d98"
server_key_version: "encryptionkey@0"
I20260809 12:04:08.025949 9455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "acc522b25d284580a2af45212bc42e40"
format_stamp: "Formatted at 2026-08-09 12:04:08 on dist-test-slave-sb2z"
server_key: "12ebb8ff9ba4f9dcdbac6e305ad82352"
server_key_iv: "e2596bc258c724262aee0b6bf6099d98"
server_key_version: "encryptionkey@0"
I20260809 12:04:08.030870 9455 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:04:08.034330 9477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:08.035624 9455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:04:08.035799 9455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "acc522b25d284580a2af45212bc42e40"
format_stamp: "Formatted at 2026-08-09 12:04:08 on dist-test-slave-sb2z"
server_key: "12ebb8ff9ba4f9dcdbac6e305ad82352"
server_key_iv: "e2596bc258c724262aee0b6bf6099d98"
server_key_version: "encryptionkey@0"
I20260809 12:04:08.035993 9455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:08.054538 9455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:08.058825 9455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:08.059171 9455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:08.060215 9455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:08.061620 9455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:08.061730 9455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:08.061888 9455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:08.061964 9455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:08.098145 9455 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:42359
I20260809 12:04:08.098176 9645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:42359 every 8 connection(s)
I20260809 12:04:08.099557 9455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:04:08.104413 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 9455
I20260809 12:04:08.104636 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:04:08.105561 589 external_mini_cluster.cc:1428] Setting key 38c192d5b18ed3f6f186441a70f20978
Aug 09 12:04:08 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277048, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:08.131338 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:08.102194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:34987 (local address 127.0.147.126:35267)
0809 12:04:08.102698 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:08.102705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:08.103772 (+ 1067us) server_negotiation.cc:407] Connection header received
0809 12:04:08.106473 (+ 2701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:08.106482 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:08.106597 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:08.106725 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:08.108806 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:08.109534 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:08.112418 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:08.113169 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:08.113315 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:08.118491 (+ 5176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:08.118498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:08.118503 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:08.118574 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:08.126487 (+ 7913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:08.127640 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:08.127648 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:08.127654 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:08.127827 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:08.128578 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:08.128585 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:08.128591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:08.129005 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:08.129163 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:08.130528 (+ 1365us) server_negotiation.cc:299] Negotiation successful
0809 12:04:08.130819 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":236,"threads_started":1}
I20260809 12:04:08.132717 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:08.102796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:34987)
0809 12:04:08.103607 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:08.103648 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:08.106111 (+ 2463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:08.107025 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:08.107035 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:08.107756 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:08.108611 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:08.108639 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:08.109708 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:08.109716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:08.112213 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:08.112233 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:08.113692 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:08.113701 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:08.114058 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:08.114904 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:08.114965 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:08.118214 (+ 3249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:08.126692 (+ 8478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:08.126704 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:08.126726 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:08.127467 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:08.127976 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:08.127984 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:08.127989 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:08.128395 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:08.129183 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:08.129199 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:08.129892 (+ 693us) client_negotiation.cc:769] Sending connection context
0809 12:04:08.130384 (+ 492us) client_negotiation.cc:240] Negotiation successful
0809 12:04:08.131103 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":271,"threads_started":1}
I20260809 12:04:08.135150 9646 heartbeater.cc:347] Connected to a master server at 127.0.147.126:35267
I20260809 12:04:08.135560 9646 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:08.136572 9646 heartbeater.cc:511] Master 127.0.147.126:35267 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.0.147.66@KRBTEST.COM; defaulting to no policy
I20260809 12:04:08.139320 9379 ts_manager.cc:194] Registered new tserver with Master: acc522b25d284580a2af45212bc42e40 (127.0.147.65:42359)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.66@KRBTEST.COM" created.
I20260809 12:04:08.141799 9379 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:34987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab.
Entry for principal oryx/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab.
I20260809 12:04:08.171751 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:35267
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab
--principal=oryx/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:08.511920 9657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:08.512199 9657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:08.512378 9657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:08.519733 9657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:08.519840 9657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:08.520049 9657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:04:08.529882 9657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab
--principal=oryx/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:35267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:08.531177 9657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:08.532567 9657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:08.546546 9665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:08.546504 9667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:08.546603 9670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:08.547622 9657 server_base.cc:1034] running on GCE node
I20260809 12:04:08.548259 9657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:08.549207 9657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:08.550434 9657 hybrid_clock.cc:648] HybridClock initialized: now 1786277048550385 us; error 50 us; skew 500 ppm
I20260809 12:04:08.550750 9657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:08 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277048, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:08.553988 9657 init.cc:377] Logged in from keytab as oryx/127.0.147.66@KRBTEST.COM (short username oryx)
I20260809 12:04:08.555465 9657 webserver.cc:460] Webserver started at http://127.0.147.66:39751/ using document root <none> and password file <none>
I20260809 12:04:08.556177 9657 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:08.556329 9657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:08.556632 9657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:08.559613 9657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "8fe60c8c3eef4f60838bb7d81fea5387"
format_stamp: "Formatted at 2026-08-09 12:04:08 on dist-test-slave-sb2z"
server_key: "95641e8f7770f2f5566e0681d268363e"
server_key_iv: "6a4d472ca7cc3d239b1c7b3aa7aa735a"
server_key_version: "encryptionkey@0"
I20260809 12:04:08.560405 9657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "8fe60c8c3eef4f60838bb7d81fea5387"
format_stamp: "Formatted at 2026-08-09 12:04:08 on dist-test-slave-sb2z"
server_key: "95641e8f7770f2f5566e0681d268363e"
server_key_iv: "6a4d472ca7cc3d239b1c7b3aa7aa735a"
server_key_version: "encryptionkey@0"
I20260809 12:04:08.565591 9657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 12:04:08.569155 9680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:08.570480 9657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:04:08.570660 9657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "8fe60c8c3eef4f60838bb7d81fea5387"
format_stamp: "Formatted at 2026-08-09 12:04:08 on dist-test-slave-sb2z"
server_key: "95641e8f7770f2f5566e0681d268363e"
server_key_iv: "6a4d472ca7cc3d239b1c7b3aa7aa735a"
server_key_version: "encryptionkey@0"
I20260809 12:04:08.570858 9657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:08.583412 9657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:08.587476 9657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:08.587826 9657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:08.588799 9657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:08.590158 9657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:08.590270 9657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:08.590415 9657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:08.590490 9657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:08.624924 9657 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:46105
I20260809 12:04:08.624974 9843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:46105 every 8 connection(s)
I20260809 12:04:08.626230 9657 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:04:08.630585 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 9657
I20260809 12:04:08.630779 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:04:08.631275 589 external_mini_cluster.cc:1428] Setting key bf4e34a55d5ad8df7c442cabf8421c14
Aug 09 12:04:08 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277048, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:08.646380 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:08.628793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:60923 (local address 127.0.147.126:35267)
0809 12:04:08.628976 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:08.628981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:08.630113 (+ 1132us) server_negotiation.cc:407] Connection header received
0809 12:04:08.632141 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:08.632150 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:08.632232 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:08.632320 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:08.634113 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:08.634824 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:08.636554 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:08.636973 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:08.637117 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:08.640689 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:08.640694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:08.640697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:08.640732 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:08.643240 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:08.644047 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:08.644054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:08.644062 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:08.644193 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:08.644641 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:08.644645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:08.644647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:08.644920 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:08.645139 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:08.645801 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 12:04:08.646036 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:04:08.647341 9846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:08.629209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.66:60923)
0809 12:04:08.629917 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:08.629961 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:08.631867 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:08.632585 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:08.632594 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:08.633217 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:08.633935 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:08.633956 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:08.634983 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:08.634990 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:08.636395 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:08.636407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:08.637206 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:08.637214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:08.637447 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:08.638148 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:08.638170 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:08.640449 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:08.643395 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:08.643403 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:08.643419 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:08.643914 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:08.644318 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:08.644323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:08.644326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:08.644511 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:08.645097 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:08.645106 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:08.645447 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 12:04:08.645701 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 12:04:08.646227 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":283,"threads_started":1}
I20260809 12:04:08.648842 9844 heartbeater.cc:347] Connected to a master server at 127.0.147.126:35267
I20260809 12:04:08.649225 9844 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:08.650128 9844 heartbeater.cc:511] Master 127.0.147.126:35267 requested a full tablet report, sending...
I20260809 12:04:08.651791 9379 ts_manager.cc:194] Registered new tserver with Master: 8fe60c8c3eef4f60838bb7d81fea5387 (127.0.147.66:46105)
I20260809 12:04:08.653082 9379 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:60923
WARNING: no policy specified for oryx/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab.
Entry for principal oryx/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab.
I20260809 12:04:08.694082 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:35267
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab
--principal=oryx/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:09.030112 9849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:09.030401 9849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:09.030499 9849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:09.037904 9849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:09.038015 9849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:09.038137 9849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:04:09.048034 9849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab
--principal=oryx/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:35267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:09.049343 9849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:09.050625 9849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:09.064181 9857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:09.064201 9856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:09.064689 9859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:09.065207 9849 server_base.cc:1034] running on GCE node
I20260809 12:04:09.065902 9849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:09.066838 9849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:09.068059 9849 hybrid_clock.cc:648] HybridClock initialized: now 1786277049068001 us; error 52 us; skew 500 ppm
I20260809 12:04:09.068370 9849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:09 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277049, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:09.071419 9849 init.cc:377] Logged in from keytab as oryx/127.0.147.67@KRBTEST.COM (short username oryx)
I20260809 12:04:09.072888 9849 webserver.cc:460] Webserver started at http://127.0.147.67:43277/ using document root <none> and password file <none>
I20260809 12:04:09.073565 9849 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:09.073712 9849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:09.074040 9849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:09.077088 9849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "d48348aed937491da6b7e827d3d7295f"
format_stamp: "Formatted at 2026-08-09 12:04:09 on dist-test-slave-sb2z"
server_key: "180d44c13222e790b897e0ac966f2310"
server_key_iv: "139340aec8d210df7a52a4624552bf3f"
server_key_version: "encryptionkey@0"
I20260809 12:04:09.077865 9849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "d48348aed937491da6b7e827d3d7295f"
format_stamp: "Formatted at 2026-08-09 12:04:09 on dist-test-slave-sb2z"
server_key: "180d44c13222e790b897e0ac966f2310"
server_key_iv: "139340aec8d210df7a52a4624552bf3f"
server_key_version: "encryptionkey@0"
I20260809 12:04:09.082885 9849 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:04:09.086355 9868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:09.087895 9849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:04:09.088039 9849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "d48348aed937491da6b7e827d3d7295f"
format_stamp: "Formatted at 2026-08-09 12:04:09 on dist-test-slave-sb2z"
server_key: "180d44c13222e790b897e0ac966f2310"
server_key_iv: "139340aec8d210df7a52a4624552bf3f"
server_key_version: "encryptionkey@0"
I20260809 12:04:09.088209 9849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:09.119628 9849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:09.124012 9849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:09.124257 9849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:09.125190 9849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:09.126430 9849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:09.126511 9849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:09.126595 9849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:09.126647 9849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:09.145820 9646 heartbeater.cc:503] Master 127.0.147.126:35267 was elected leader, sending a full tablet report...
I20260809 12:04:09.160516 9849 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:34885
I20260809 12:04:09.160557 10029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:34885 every 8 connection(s)
I20260809 12:04:09.161813 9849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:04:09.170681 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 9849
I20260809 12:04:09.170876 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:04:09.171352 589 external_mini_cluster.cc:1428] Setting key 32276eeb1808cdba92bdca86bc45093a
Aug 09 12:04:09 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277049, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:09.183511 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.164446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:34793 (local address 127.0.147.126:35267)
0809 12:04:09.164987 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:09.164992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:09.165803 (+ 811us) server_negotiation.cc:407] Connection header received
0809 12:04:09.167841 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:09.167847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:09.167937 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:09.168071 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:09.169734 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.170539 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.172409 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.173298 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.173426 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.177090 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:09.177095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:09.177098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:09.177128 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:09.180722 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.181411 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.181415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.181417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.181536 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.181900 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.181903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.181905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.182194 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:09.182305 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:09.182927 (+ 622us) server_negotiation.cc:299] Negotiation successful
0809 12:04:09.183120 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":300,"threads_started":1}
I20260809 12:04:09.184295 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.164930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.67:34793)
0809 12:04:09.165624 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:09.165668 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:09.167574 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:09.168296 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:09.168303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:09.168796 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:09.169554 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.169581 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.170703 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.170707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.172221 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.172238 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.173534 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.173543 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.173811 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.174598 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:09.174620 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:09.176905 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:09.180870 (+ 3965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.180876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.180887 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.181291 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.181628 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.181631 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.181633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.181803 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.182297 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:09.182302 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:09.182587 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 12:04:09.182795 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 12:04:09.183148 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":288,"threads_started":1}
I20260809 12:04:09.185927 10031 heartbeater.cc:347] Connected to a master server at 127.0.147.126:35267
I20260809 12:04:09.186291 10031 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:09.187224 10031 heartbeater.cc:511] Master 127.0.147.126:35267 requested a full tablet report, sending...
I20260809 12:04:09.188681 9379 ts_manager.cc:194] Registered new tserver with Master: d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885)
I20260809 12:04:09.189612 9379 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:34793
I20260809 12:04:09.199200 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.147.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.125.keytab.
Entry for principal oryx/127.0.147.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.125.keytab.
I20260809 12:04:09.262557 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.125:35491
--webserver_interface=127.0.147.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.125.keytab
--principal=oryx/127.0.147.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:35267,127.0.147.125:35491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:09.605235 10043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:09.605522 10043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:09.605613 10043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:09.612823 10043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:09.612932 10043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:09.612980 10043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:09.613042 10043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:09.613081 10043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:09.623018 10043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:35267,127.0.147.125:35491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.125.keytab
--principal=oryx/127.0.147.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.125:35491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.147.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
W20260809 12:04:09.623397 10043 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.147.126:35267,127.0.147.125:35491'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 12:04:09.624663 10043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:09.626375 10043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:09.640801 10049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:09.641142 10053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:09.642820 10043 server_base.cc:1034] running on GCE node
W20260809 12:04:09.643131 10050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:09.644300 10043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:09.645798 10043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:09.647078 10043 hybrid_clock.cc:648] HybridClock initialized: now 1786277049646969 us; error 78 us; skew 500 ppm
I20260809 12:04:09.647464 10043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:09 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277049, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:09.654824 10043 init.cc:377] Logged in from keytab as oryx/127.0.147.125@KRBTEST.COM (short username oryx)
I20260809 12:04:09.656558 9844 heartbeater.cc:503] Master 127.0.147.126:35267 was elected leader, sending a full tablet report...
I20260809 12:04:09.656927 10043 webserver.cc:460] Webserver started at http://127.0.147.125:35055/ using document root <none> and password file <none>
I20260809 12:04:09.657836 10043 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:09.657955 10043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:09.658531 10043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:09.662228 10043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data/instance:
uuid: "30cfede53d33410e951ee4fbdccb0205"
format_stamp: "Formatted at 2026-08-09 12:04:09 on dist-test-slave-sb2z"
server_key: "b9089865e5b959b8456da66c50a83c56"
server_key_iv: "e9c537fb5225f800f017f38aa8b94cac"
server_key_version: "encryptionkey@0"
I20260809 12:04:09.663174 10043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal/instance:
uuid: "30cfede53d33410e951ee4fbdccb0205"
format_stamp: "Formatted at 2026-08-09 12:04:09 on dist-test-slave-sb2z"
server_key: "b9089865e5b959b8456da66c50a83c56"
server_key_iv: "e9c537fb5225f800f017f38aa8b94cac"
server_key_version: "encryptionkey@0"
I20260809 12:04:09.669090 10043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:04:09.673307 10066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:09.674991 10043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 12:04:09.675251 10043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal
uuid: "30cfede53d33410e951ee4fbdccb0205"
format_stamp: "Formatted at 2026-08-09 12:04:09 on dist-test-slave-sb2z"
server_key: "b9089865e5b959b8456da66c50a83c56"
server_key_iv: "e9c537fb5225f800f017f38aa8b94cac"
server_key_version: "encryptionkey@0"
I20260809 12:04:09.675617 10043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:09.709740 10043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:09.714561 10043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:09.714862 10043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:09.735569 10043 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.125:35491
I20260809 12:04:09.735555 10145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.125:35491 every 8 connection(s)
I20260809 12:04:09.736881 10043 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
I20260809 12:04:09.738942 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 10043
I20260809 12:04:09.741598 10147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:04:09 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.125@KRBTEST.COM
I20260809 12:04:09.752642 10147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:04:09.769277 10153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.741447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.0.147.125:35491)
0809 12:04:09.742310 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:09.742321 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:09.742363 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:04:09.744041 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:09.744063 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:09.744439 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:09.744851 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:09.746424 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.747945 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.749210 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.749879 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.750092 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.754991 (+ 4899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:09.754999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:09.755013 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:09.755049 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:09.762732 (+ 7683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.763705 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.763714 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.763729 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.763890 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.764403 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.764409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.764413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.765337 (+ 924us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:09.765868 (+ 531us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:09.766513 (+ 645us) server_negotiation.cc:299] Negotiation successful
0809 12:04:09.767024 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":651,"thread_start_us":430,"threads_started":1}
Aug 09 12:04:09 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277049, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.125@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:09.781714 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.756386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.0.147.126:35267)
0809 12:04:09.759226 (+ 2840us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:09.759235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:09.759256 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:04:09.759318 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:09.759324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:09.759435 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:09.759589 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:09.762365 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.764457 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.766663 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.767348 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.767474 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.773106 (+ 5632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:09.773111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:09.773115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:09.773152 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:09.776902 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.779172 (+ 2270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.779176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.779178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.779397 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.779945 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.779949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.779951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.780397 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:09.780527 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:09.781155 (+ 628us) server_negotiation.cc:299] Negotiation successful
0809 12:04:09.781358 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2689,"thread_start_us":2505,"threads_started":1}
I20260809 12:04:09.781819 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.756383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:48418)
0809 12:04:09.757907 (+ 1524us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:09.757961 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:09.758176 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:09.759641 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:09.759648 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:09.761341 (+ 1693us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:09.762134 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.762150 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.764689 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.764695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.766488 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.766501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.767695 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.767704 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.768133 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.769203 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:09.769246 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:09.772914 (+ 3668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:09.778464 (+ 5550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.778474 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.778480 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.779042 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.779592 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.779602 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.779605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.779800 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.780533 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:09.780539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:09.780716 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 12:04:09.780981 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 12:04:09.781285 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"mutex_wait_us":3,"thread_start_us":804,"threads_started":1}
I20260809 12:04:09.784016 10147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:04:09 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277049, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.125@KRBTEST.COM for oryx/127.0.147.125@KRBTEST.COM
I20260809 12:04:09.798928 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.785071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.0.1:50762)
0809 12:04:09.785295 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:09.785314 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:09.785415 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:09.785664 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:09.785666 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:09.786203 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:09.786427 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.786436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.787892 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.787896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.789040 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.789052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.789903 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.789908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.790113 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.790813 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:09.790829 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:09.793665 (+ 2836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:09.796751 (+ 3086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.796754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.796757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.797152 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.797496 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.797499 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.797501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.797634 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.798249 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:09.798252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:09.798371 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 12:04:09.798484 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 12:04:09.798695 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 12:04:09.799329 10153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.785046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.0.147.125:35491)
0809 12:04:09.785221 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:09.785225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:09.785383 (+ 158us) server_negotiation.cc:407] Connection header received
0809 12:04:09.785498 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:09.785501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:09.785556 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:09.785628 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:09.786605 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.787717 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.789168 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.789732 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.789825 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.793818 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:09.793823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:09.793826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:09.793853 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:09.796639 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.797253 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.797257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.797259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.797398 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.797733 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.797737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.797741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.798144 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:09.798255 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:09.798640 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 12:04:09.798918 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:04:09.804463 10147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Bootstrap starting.
I20260809 12:04:09.808255 10147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:09.809787 10147 log.cc:824] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:09.813225 10147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: No bootstrap required, opened a new log
I20260809 12:04:09.824156 10147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } }
I20260809 12:04:09.824657 10147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:09.824800 10147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30cfede53d33410e951ee4fbdccb0205, State: Initialized, Role: FOLLOWER
I20260809 12:04:09.825536 10147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } }
I20260809 12:04:09.826865 10158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } } }
I20260809 12:04:09.827487 10158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:04:09.827867 10147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 12:04:09.844087 10169 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:04:09.844237 10169 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:04:09.847579 10043 master_runner.cc:428] Detected that this master 30cfede53d33410e951ee4fbdccb0205 is joining an existing cluster
I20260809 12:04:09.847834 10043 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.147.125:35491
I20260809 12:04:09.869014 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.852538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.0.147.126:35267)
0809 12:04:09.852743 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:09.852750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:09.853687 (+ 937us) server_negotiation.cc:407] Connection header received
0809 12:04:09.854039 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:09.854048 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:09.854147 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:09.854260 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:09.855862 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.857112 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.858946 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.859410 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.859570 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.861854 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:09.861859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:09.861863 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:09.861894 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:09.865588 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.866603 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.866608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.866612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.866777 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.867241 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.867248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.867252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.867677 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:09.867849 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:09.868380 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 12:04:09.868626 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:04:09.869519 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.852649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:48420)
0809 12:04:09.853569 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:09.853596 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:09.853809 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:09.854306 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:09.854310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:09.855206 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:09.855673 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.855692 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.857270 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.857275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.858816 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.858827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.859828 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.859835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.859976 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.860623 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:09.860649 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:09.861653 (+ 1004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:09.865742 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.865748 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.865753 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.866482 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.866925 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.866931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.866935 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.867121 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.867812 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:09.867818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:09.868051 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 12:04:09.868279 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 12:04:09.868628 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":644,"threads_started":1}
I20260809 12:04:09.869721 10153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.852890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.0.147.125:35491)
0809 12:04:09.853124 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:09.853132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:09.854122 (+ 990us) server_negotiation.cc:407] Connection header received
0809 12:04:09.854305 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:09.854310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:09.854373 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:09.854459 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:09.855975 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.856658 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.858671 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.859474 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.859660 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.862396 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:09.862403 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:09.862409 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:09.862446 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:09.866372 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.867137 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.867144 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.867149 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.867375 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.867831 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.867835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.867838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.868133 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:09.868295 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:09.868864 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 12:04:09.869084 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:04:09.870057 10179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.852890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.0.1:50774)
0809 12:04:09.854019 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:09.854037 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:09.854173 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:09.854690 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:09.854694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:09.855422 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:09.855848 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.855859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.856828 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.856833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.858503 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.858518 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.859954 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.859962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.860177 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.860831 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:09.860857 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:09.862186 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:09.866503 (+ 4317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.866509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.866513 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.867030 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.867503 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.867508 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.867511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.867718 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.868454 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:09.868458 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:09.868618 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 12:04:09.869256 (+ 638us) client_negotiation.cc:240] Negotiation successful
0809 12:04:09.869539 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"mutex_wait_us":1,"thread_start_us":784,"threads_started":1}
Aug 09 12:04:09 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277047, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.126@KRBTEST.COM for oryx/127.0.147.125@KRBTEST.COM
I20260809 12:04:09.894866 10153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.876675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50778 (local address 127.0.147.125:35491)
0809 12:04:09.876857 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:09.876863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:09.877583 (+ 720us) server_negotiation.cc:407] Connection header received
0809 12:04:09.877985 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:09.877993 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:09.878091 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:09.878231 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:09.879844 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.881029 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.883265 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.884083 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.884230 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.888656 (+ 4426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:09.888662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:09.888665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:09.888694 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:09.891843 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.892772 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.892778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.892782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.892925 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:09.893325 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:09.893330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:09.893334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:09.893640 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:09.893796 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:09.894249 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 12:04:09.894484 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:04:09.894919 10180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:09.876680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.0.1:50778)
0809 12:04:09.877435 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:09.877493 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:09.877748 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:09.878292 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:09.878301 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:09.879260 (+ 959us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:09.879626 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.879648 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.881188 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.881195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.883087 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:09.883107 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:09.884281 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:09.884288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:09.884530 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:09.885314 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:09.885341 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:09.888449 (+ 3108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:09.892053 (+ 3604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.892063 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.892069 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.892582 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.893041 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:09.893047 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:09.893051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:09.893214 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:09.893773 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:09.893781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:09.893965 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 12:04:09.894154 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 12:04:09.894484 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":314,"threads_started":1}
I20260809 12:04:09.897370 9379 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.147.125:35491: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "30cfede53d33410e951ee4fbdccb0205"
member_type: NON_VOTER
last_known_addr {
host: "127.0.147.125"
port: 35491
}
attrs {
promote: true
}
}
dest_uuid: "7b99ca339e2e440ba634712c2a35f0bf"
cas_config_opid_index: -1
I20260809 12:04:09.898792 9379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.0.147.124@KRBTEST.COM; defaulting to no policy
I20260809 12:04:09.907210 10106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:09.908172 10186 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 30cfede53d33410e951ee4fbdccb0205 (127.0.147.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.124@KRBTEST.COM" created.
I20260809 12:04:09.911628 10106 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Refusing update from remote peer 7b99ca339e2e440ba634712c2a35f0bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 12:04:09.912900 10184 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:09.923775 10186 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.147.125:35491
I20260809 12:04:09.924899 10043 master.cc:537] Master@127.0.147.125:35491 shutting down...
I20260809 12:04:09.924953 10184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:09.926162 10184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:09.946219 10106 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 30cfede53d33410e951ee4fbdccb0205 (127.0.147.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.124.keytab.
Entry for principal oryx/127.0.147.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.124.keytab.
I20260809 12:04:09.953620 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.124:43561
--webserver_interface=127.0.147.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.124.keytab
--principal=oryx/127.0.147.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:35267,127.0.147.125:35491,127.0.147.124:43561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:04:09.977810 10204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:09.978205 10204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:04:09.978338 10197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:09.978806 10197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:04:09.978665 10158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b99ca339e2e440ba634712c2a35f0bf. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:09.978924 10158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:04:09.982434 10043 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 LEARNER]: Raft consensus shutting down.
I20260809 12:04:09.983057 10043 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 LEARNER]: Raft consensus is shut down!
I20260809 12:04:09.983253 10043 tablet_replica.cc:330] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: stopping tablet replica
I20260809 12:04:10.003039 10043 master.cc:552] Master@127.0.147.125:35491 shutdown complete.
I20260809 12:04:10.003194 10043 master_runner.cc:305] Clearing existing system tablet
I20260809 12:04:10.003765 10043 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:10.003878 10043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:10.012369 10043 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.000s sys 0.008s
I20260809 12:04:10.015998 10206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:10.016537 10043 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:10.017032 10043 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260809 12:04:10.017205 10043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal
uuid: "30cfede53d33410e951ee4fbdccb0205"
format_stamp: "Formatted at 2026-08-09 12:04:09 on dist-test-slave-sb2z"
server_key: "b9089865e5b959b8456da66c50a83c56"
server_key_iv: "e9c537fb5225f800f017f38aa8b94cac"
server_key_version: "encryptionkey@0"
I20260809 12:04:10.022614 10043 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:10.031599 10043 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 12:04:10.031914 10043 log.cc:1196] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 12:04:10.032572 10043 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Deleting consensus metadata
I20260809 12:04:10.032912 10043 master_runner.cc:315] Copying system tablet from 127.0.147.126:35267
I20260809 12:04:10.037755 10043 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: tablet copy: Beginning tablet copy session from remote peer at address 127.0.147.126:35267
I20260809 12:04:10.058208 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.038901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.0.147.126:35267)
0809 12:04:10.039126 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.039132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.039782 (+ 650us) server_negotiation.cc:407] Connection header received
0809 12:04:10.040108 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.040114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.040202 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.040307 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.041665 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.042781 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.044729 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.045646 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.045749 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.049386 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.049393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.049410 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.049457 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.054895 (+ 5438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.055774 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.055779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.055783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.055947 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.056454 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.056459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.056463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.056826 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.056960 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.057601 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.057828 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:04:10.059638 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.038835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:48432)
0809 12:04:10.039651 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.039675 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.039907 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.040602 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.040607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.041180 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:10.041478 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.041492 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.043000 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.043005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.044555 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.044571 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.046109 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.046115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.046403 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.047217 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:10.047241 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:10.049086 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:10.055089 (+ 6003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.055094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.055097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.055632 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.056151 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.056156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.056159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.056320 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.057063 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:10.057068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:10.057287 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 12:04:10.058768 (+ 1481us) client_negotiation.cc:240] Negotiation successful
0809 12:04:10.059169 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":503,"threads_started":1}
I20260809 12:04:10.061414 9401 tablet_copy_service.cc:140] P 7b99ca339e2e440ba634712c2a35f0bf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 30cfede53d33410e951ee4fbdccb0205 ({username='oryx', principal='oryx/127.0.147.125@KRBTEST.COM'} at 127.0.0.1:48432)
I20260809 12:04:10.061825 9401 tablet_copy_service.cc:161] P 7b99ca339e2e440ba634712c2a35f0bf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 30cfede53d33410e951ee4fbdccb0205 at {username='oryx', principal='oryx/127.0.147.125@KRBTEST.COM'} at 127.0.0.1:48432: session id = 30cfede53d33410e951ee4fbdccb0205-00000000000000000000000000000000
I20260809 12:04:10.066808 9401 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 12:04:10.072654 10043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 12:04:10.081512 10043 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: tablet copy: Starting download of 0 data blocks...
I20260809 12:04:10.081877 10043 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: tablet copy: Starting download of 1 WAL segments...
I20260809 12:04:10.085778 10043 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 12:04:10.090865 9401 tablet_copy_service.cc:342] P 7b99ca339e2e440ba634712c2a35f0bf: Request end of tablet copy session 30cfede53d33410e951ee4fbdccb0205-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.147.125@KRBTEST.COM'} at 127.0.0.1:48432
I20260809 12:04:10.091372 9401 tablet_copy_service.cc:433] P 7b99ca339e2e440ba634712c2a35f0bf: ending tablet copy session 30cfede53d33410e951ee4fbdccb0205-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 30cfede53d33410e951ee4fbdccb0205
I20260809 12:04:10.105262 10043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:10.111815 10215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:10.113854 10216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:10.123379 10219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:10.129144 10043 server_base.cc:1034] running on GCE node
I20260809 12:04:10.129721 10043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:10.131798 10043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:10.133059 10043 hybrid_clock.cc:648] HybridClock initialized: now 1786277050132977 us; error 54 us; skew 500 ppm
I20260809 12:04:10.133306 10043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:10 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277050, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:10.136051 10043 init.cc:377] Logged in from keytab as oryx/127.0.147.125@KRBTEST.COM (short username oryx)
I20260809 12:04:10.139401 10043 webserver.cc:460] Webserver started at http://127.0.147.125:38887/ using document root <none> and password file <none>
I20260809 12:04:10.139828 10043 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:10.139930 10043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:10.143290 10043 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 12:04:10.145632 10229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:10.146175 10043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 12:04:10.146361 10043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal
uuid: "30cfede53d33410e951ee4fbdccb0205"
format_stamp: "Formatted at 2026-08-09 12:04:09 on dist-test-slave-sb2z"
server_key: "b9089865e5b959b8456da66c50a83c56"
server_key_iv: "e9c537fb5225f800f017f38aa8b94cac"
server_key_version: "encryptionkey@0"
I20260809 12:04:10.146571 10043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:10.180389 10043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:10.185286 10043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:10.192924 10031 heartbeater.cc:503] Master 127.0.147.126:35267 was elected leader, sending a full tablet report...
I20260809 12:04:10.239387 10043 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.125:35491
I20260809 12:04:10.239457 10295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.125:35491 every 8 connection(s)
I20260809 12:04:10.240541 10043 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
I20260809 12:04:10.246634 10297 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:04:10.250625 10297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Bootstrap starting.
I20260809 12:04:10.266942 10297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:04:10.267606 10297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Bootstrap complete.
I20260809 12:04:10.268829 10297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } }
I20260809 12:04:10.269097 10297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 30cfede53d33410e951ee4fbdccb0205, State: Initialized, Role: LEARNER
I20260809 12:04:10.269349 10297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } }
I20260809 12:04:10.270179 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:10.270556 10297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:10.270674 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:04:10.284408 10316 catalog_manager.cc:1259] Loaded cluster ID: 6fe03249f99e4815a346c82328311b7a
I20260809 12:04:10.284538 10316 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: loading cluster ID for follower catalog manager: success
I20260809 12:04:10.287297 10316 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: acquiring CA information for follower catalog manager: success
I20260809 12:04:10.289300 10316 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:04:10.397656 10186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:10.397979 10186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:10.413311 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.397974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.0.1:50794)
0809 12:04:10.404251 (+ 6277us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.404278 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.404472 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.404997 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.405005 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.405041 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:10.405310 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.405326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.407417 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.407425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.410091 (+ 2666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.410111 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.412130 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.412139 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.412421 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.412500 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 12:04:10.412599 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 12:04:10.412770 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6162,"thread_start_us":6026,"threads_started":1}
I20260809 12:04:10.413479 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.398190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50794 (local address 127.0.147.125:35491)
0809 12:04:10.404083 (+ 5893us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.404091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.404429 (+ 338us) server_negotiation.cc:407] Connection header received
0809 12:04:10.404632 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.404640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.404759 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.404929 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:10.405515 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.407227 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.410294 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.411905 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.412082 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.412490 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.412648 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.412856 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5717,"thread_start_us":5540,"threads_started":1}
I20260809 12:04:10.415414 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b99ca339e2e440ba634712c2a35f0bf. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:10.415894 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:04:10.416909 10186 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: attempting to promote NON_VOTER 30cfede53d33410e951ee4fbdccb0205 to VOTER
I20260809 12:04:10.418498 10186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } }
I20260809 12:04:10.420948 10264 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 LEARNER]: Refusing update from remote peer 7b99ca339e2e440ba634712c2a35f0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 12:04:10.421797 10186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 12:04:10.426525 10184 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 30cfede53d33410e951ee4fbdccb0205 (127.0.147.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } }
I20260809 12:04:10.428433 10264 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 30cfede53d33410e951ee4fbdccb0205 (127.0.147.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } }
I20260809 12:04:10.432588 10184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } }
I20260809 12:04:10.432920 10184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:10.433341 10187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } }
I20260809 12:04:10.434085 10187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:10.435360 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } }
I20260809 12:04:10.435724 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 12:04:10.448973 10202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:10.449357 10202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:10.449591 10202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:10.460543 10202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:10.460799 10202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:10.461004 10202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:10.461170 10202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:10.461254 10202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:10.475700 10202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:35267,127.0.147.125:35491,127.0.147.124:43561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.124.keytab
--principal=oryx/127.0.147.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.124:43561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.147.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:10.477203 10202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:10.478663 10202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:10.496142 10327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:10.497285 10333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:10.497334 10330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:10.498313 10202 server_base.cc:1034] running on GCE node
I20260809 12:04:10.499261 10202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:10.500931 10202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:10.502242 10202 hybrid_clock.cc:648] HybridClock initialized: now 1786277050502162 us; error 32 us; skew 500 ppm
I20260809 12:04:10.502713 10202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:10 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277050, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:10.506939 10202 init.cc:377] Logged in from keytab as oryx/127.0.147.124@KRBTEST.COM (short username oryx)
I20260809 12:04:10.509069 10202 webserver.cc:460] Webserver started at http://127.0.147.124:45143/ using document root <none> and password file <none>
I20260809 12:04:10.509867 10202 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:10.510102 10202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:10.510452 10202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:10.514268 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data/instance:
uuid: "0305fc037f3e402b83e867dc3bc2b91f"
format_stamp: "Formatted at 2026-08-09 12:04:10 on dist-test-slave-sb2z"
server_key: "aecd979c3202b91ef93ac78dbec519b7"
server_key_iv: "d13cd9d2acf758d66b83154500561188"
server_key_version: "encryptionkey@0"
I20260809 12:04:10.515204 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal/instance:
uuid: "0305fc037f3e402b83e867dc3bc2b91f"
format_stamp: "Formatted at 2026-08-09 12:04:10 on dist-test-slave-sb2z"
server_key: "aecd979c3202b91ef93ac78dbec519b7"
server_key_iv: "d13cd9d2acf758d66b83154500561188"
server_key_version: "encryptionkey@0"
I20260809 12:04:10.521433 10202 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260809 12:04:10.525947 10341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:10.527405 10202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:10.527670 10202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal
uuid: "0305fc037f3e402b83e867dc3bc2b91f"
format_stamp: "Formatted at 2026-08-09 12:04:10 on dist-test-slave-sb2z"
server_key: "aecd979c3202b91ef93ac78dbec519b7"
server_key_iv: "d13cd9d2acf758d66b83154500561188"
server_key_version: "encryptionkey@0"
I20260809 12:04:10.527918 10202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:10.564368 10202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:10.570303 10202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:10.570789 10202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:10.613933 10202 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.124:43561
I20260809 12:04:10.614071 10412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.124:43561 every 8 connection(s)
I20260809 12:04:10.615825 10202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
I20260809 12:04:10.620671 10413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:10.621707 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 10202
I20260809 12:04:10.633497 10413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:04:10 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.124@KRBTEST.COM
Aug 09 12:04:10 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277050, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.124@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:10.656160 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.624214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38932 (local address 127.0.147.124:43561)
0809 12:04:10.625603 (+ 1389us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.625615 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.625677 (+ 62us) server_negotiation.cc:407] Connection header received
0809 12:04:10.629345 (+ 3668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.629369 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.629786 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.630244 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.633273 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.634965 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.638323 (+ 3358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.639708 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.639897 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.646378 (+ 6481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.646388 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.646401 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.646535 (+ 134us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.650982 (+ 4447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.651966 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.651972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.651980 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.652185 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.652746 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.652750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.652752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.653360 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.653703 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.654277 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.654820 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1135,"thread_start_us":876,"threads_started":1}
I20260809 12:04:10.659114 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.637036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.0.147.126:35267)
0809 12:04:10.638049 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.638066 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.639016 (+ 950us) server_negotiation.cc:407] Connection header received
0809 12:04:10.639325 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.639329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.639397 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.639613 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.641492 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.643205 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.645398 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.645966 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.646169 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.651081 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.651086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.651089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.651121 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.655389 (+ 4268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.656424 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.656429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.656432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.656677 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.657169 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.657175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.657183 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.657535 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.657741 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.658323 (+ 582us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.658570 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":627,"threads_started":1}
I20260809 12:04:10.659168 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.637111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:48436)
0809 12:04:10.638696 (+ 1585us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.638750 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.639099 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.639667 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.639675 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.640721 (+ 1046us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:10.641306 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.641317 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.643372 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.643376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.645125 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.645165 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.646327 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.646341 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.646616 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.647710 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:10.647738 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:10.650830 (+ 3092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:10.655567 (+ 4737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.655578 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.655585 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.656254 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.656797 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.656802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.656807 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.657037 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.657735 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:10.657744 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:10.657948 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 12:04:10.658214 (+ 266us) client_negotiation.cc:240] Negotiation successful
0809 12:04:10.658543 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1280,"thread_start_us":1015,"threads_started":1}
I20260809 12:04:10.661319 10413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:04:10 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277050, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.124@KRBTEST.COM for oryx/127.0.147.125@KRBTEST.COM
I20260809 12:04:10.676816 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.662073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.0.1:50798)
0809 12:04:10.662360 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.662382 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.662517 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.662819 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.662822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.663321 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:10.663614 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.663626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.664821 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.664826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.666045 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.666059 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.666863 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.666869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.667055 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.667809 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:10.667826 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:10.670554 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:10.674469 (+ 3915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.674473 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.674476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.675092 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.675472 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.675475 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.675477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.675660 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.676147 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:10.676150 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:10.676273 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 12:04:10.676372 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 12:04:10.676585 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 12:04:10.676920 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.662370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50798 (local address 127.0.147.125:35491)
0809 12:04:10.662558 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.662564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.662582 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:04:10.662632 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.662636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.662694 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.662775 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.663760 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.664684 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.666214 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.666688 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.666784 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.670776 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.670787 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.670791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.670825 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.674352 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.675194 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.675197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.675200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.675374 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.675754 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.675757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.675759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.676039 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.676160 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.676480 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.676647 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:04:10.677946 10413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:04:10 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277050, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.124@KRBTEST.COM for oryx/127.0.147.124@KRBTEST.COM
I20260809 12:04:10.690248 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.678835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:43561 (local address 127.0.0.1:38940)
0809 12:04:10.678981 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.678997 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.679088 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.679320 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.679323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.679710 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:10.679848 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.679856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.680779 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.680783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.681602 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.681613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.682224 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.682228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.682384 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.682894 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:10.682909 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:10.684914 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:10.688267 (+ 3353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.688271 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.688273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.688616 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.688913 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.688917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.688919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.689069 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.689518 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:10.689521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:10.689656 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 12:04:10.689762 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:04:10.689941 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 12:04:10.690357 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.678715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.0.147.124:43561)
0809 12:04:10.678909 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.678914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.679069 (+ 155us) server_negotiation.cc:407] Connection header received
0809 12:04:10.679159 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.679162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.679214 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.679286 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.679964 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.680663 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.681719 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.682104 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.682196 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.685088 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.685093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.685096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.685123 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.688148 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.688713 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.688717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.688719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.688822 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.689160 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.689163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.689165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.689422 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.689531 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.689846 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.690080 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":67}
I20260809 12:04:10.695740 10413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Bootstrap starting.
I20260809 12:04:10.699239 10413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:10.700353 10413 log.cc:824] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:10.703064 10413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: No bootstrap required, opened a new log
I20260809 12:04:10.711211 10413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } }
I20260809 12:04:10.711835 10413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:10.712029 10413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0305fc037f3e402b83e867dc3bc2b91f, State: Initialized, Role: FOLLOWER
I20260809 12:04:10.712587 10413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } }
I20260809 12:04:10.713747 10420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } } }
I20260809 12:04:10.714226 10420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:04:10.714969 10413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: configured and running, proceeding with master startup.
W20260809 12:04:10.729043 10436 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:04:10.729272 10436 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:04:10.733877 10202 master_runner.cc:428] Detected that this master 0305fc037f3e402b83e867dc3bc2b91f is joining an existing cluster
I20260809 12:04:10.734215 10202 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.147.124:43561
I20260809 12:04:10.764293 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 9455
I20260809 12:04:10.772154 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.740819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48450 (local address 127.0.147.126:35267)
0809 12:04:10.741047 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.741055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.752031 (+ 10976us) server_negotiation.cc:407] Connection header received
0809 12:04:10.752728 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.752737 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.752911 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.753000 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.754831 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.755861 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.759596 (+ 3735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.760292 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.760432 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.763269 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.763275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.763280 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.763325 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.768342 (+ 5017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.769239 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.769245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.769249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.769418 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.769992 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.769997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.770000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.770466 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.770629 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.771551 (+ 922us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.771815 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:04:10.779400 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.742179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:48450)
0809 12:04:10.751787 (+ 9608us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.751818 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.752492 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.753261 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.753264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.754183 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:10.754658 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.754669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.756107 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.756117 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.759391 (+ 3274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.759407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.760543 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.760549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.760737 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.761643 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:10.761661 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:10.763082 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:10.768533 (+ 5451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.768539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.768544 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.769096 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.769604 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.769609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.769613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.769844 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.770728 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:10.770734 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:10.771127 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 12:04:10.772430 (+ 1303us) client_negotiation.cc:240] Negotiation successful
0809 12:04:10.776052 (+ 3622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9307,"thread_start_us":17451,"threads_started":1}
Aug 09 12:04:10 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.65@KRBTEST.COM
I20260809 12:04:10.811484 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.768389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39778 (local address 127.0.147.65:42359)
0809 12:04:10.778628 (+ 10239us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.778640 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.778672 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:04:10.778741 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.778750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.779024 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.779462 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.784307 (+ 4845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.786732 (+ 2425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.789832 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.791682 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.792975 (+ 1293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.799625 (+ 6650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.799647 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.799669 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.799720 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.805687 (+ 5967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.806463 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.806474 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.806477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.806707 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.807290 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.807296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.807300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.808035 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.808460 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.810259 (+ 1799us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.810662 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10039,"thread_start_us":8073,"threads_started":1}
I20260809 12:04:10.813848 9638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:10.815488 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.741514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50806 (local address 127.0.147.125:35491)
0809 12:04:10.741722 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.741727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.776466 (+ 34739us) server_negotiation.cc:407] Connection header received
0809 12:04:10.776954 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.776961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.777101 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.777234 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.782146 (+ 4912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.783996 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.788766 (+ 4770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.789646 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.789767 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.798367 (+ 8600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.798375 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.798379 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.798460 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.804471 (+ 6011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.807811 (+ 3340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.807817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.807821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.808046 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.808883 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.808889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.808892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.809307 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.809451 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.814478 (+ 5027us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.814833 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:04:10.818778 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.742685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.0.1:50806)
0809 12:04:10.776167 (+ 33482us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.776187 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.776650 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.777309 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.777330 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.778970 (+ 1640us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:10.780436 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.780453 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.784175 (+ 3722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.784182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.788581 (+ 4399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.788596 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.790190 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.790202 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.790554 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.795834 (+ 5280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:10.795873 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:10.798052 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:10.804734 (+ 6682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.804754 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.804760 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.807621 (+ 2861us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.808199 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.808206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.808209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.808705 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.812967 (+ 4262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:10.812974 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:10.813698 (+ 724us) client_negotiation.cc:769] Sending connection context
0809 12:04:10.813850 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 12:04:10.817764 (+ 3914us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33264,"mutex_wait_us":2,"thread_start_us":10294,"threads_started":1}
I20260809 12:04:10.823215 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.741412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38946 (local address 127.0.147.124:43561)
0809 12:04:10.741656 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.741661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.758723 (+ 17062us) server_negotiation.cc:407] Connection header received
0809 12:04:10.759008 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.759016 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.759101 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.759181 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.779861 (+ 20680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.781898 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.784844 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.785884 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.786026 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.789743 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.789750 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.789755 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.789792 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.812178 (+ 22386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.819412 (+ 7234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.819428 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.819433 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.819738 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.820622 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.820627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.820631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.821194 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.821424 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.822012 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.822440 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":67}
I20260809 12:04:10.823734 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.742457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:43561 (local address 127.0.0.1:38946)
0809 12:04:10.758519 (+ 16062us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.758548 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.758842 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.759443 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.759451 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.778148 (+ 18697us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:10.779691 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.779704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.782174 (+ 2470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.782179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.784637 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.784674 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.786524 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.786531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.786829 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.787752 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:10.787776 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:10.789557 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:10.812323 (+ 22766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.812332 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.812337 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.814577 (+ 2240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.820156 (+ 5579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.820165 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.820170 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.820427 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.821353 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:10.821374 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:10.821584 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 12:04:10.821796 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 12:04:10.822201 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15580,"mutex_wait_us":50,"thread_start_us":8313,"threads_started":1}
W20260809 12:04:10.824903 10353 tablet.cc:2369] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 12:04:10 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277047, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.126@KRBTEST.COM for oryx/127.0.147.124@KRBTEST.COM
I20260809 12:04:10.845180 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.792717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:43561 (local address 127.0.0.1:38956)
0809 12:04:10.792994 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.793025 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.793148 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.820761 (+ 27613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.820770 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.821484 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:10.821810 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.821829 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.828188 (+ 6359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.828195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.830022 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.830043 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.831327 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.831335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.831617 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.832702 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:10.832727 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:10.836855 (+ 4128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:10.842013 (+ 5158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.842019 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.842023 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.842561 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.843015 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:10.843020 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:10.843026 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:10.843224 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:10.843998 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:10.844005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:10.844212 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 12:04:10.844390 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 12:04:10.844695 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 12:04:10.845216 10446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.793080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.0.147.124:43561)
0809 12:04:10.820055 (+ 26975us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.820060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.820092 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:04:10.820187 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.820192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.820284 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.821964 (+ 1680us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.826273 (+ 4309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.827980 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.830223 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.831122 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.831281 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.837232 (+ 5951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.837242 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.837247 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.837289 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:10.841846 (+ 4557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.842704 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.842711 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.842715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.842867 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:10.843362 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:10.843367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:10.843372 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:10.843841 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:10.843991 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:10.844539 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 12:04:10.844767 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26219,"thread_start_us":26802,"threads_started":1}
I20260809 12:04:10.847800 9379 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.147.124:43561: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f"
member_type: NON_VOTER
last_known_addr {
host: "127.0.147.124"
port: 43561
}
attrs {
promote: true
}
}
dest_uuid: "7b99ca339e2e440ba634712c2a35f0bf"
cas_config_opid_index: 6
I20260809 12:04:10.848802 9379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } }
I20260809 12:04:10.851462 10264 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Refusing update from remote peer 7b99ca339e2e440ba634712c2a35f0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 12:04:10.852229 10381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:10.852249 10187 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 12:04:10.855557 10187 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } } }
I20260809 12:04:10.858151 10381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 FOLLOWER]: Refusing update from remote peer 7b99ca339e2e440ba634712c2a35f0bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 12:04:10.859697 10186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:10.862710 10264 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } } }
I20260809 12:04:10.864132 10187 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.147.124:43561
I20260809 12:04:10.864547 10187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } } }
I20260809 12:04:10.865790 10187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:10.866405 10186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } } }
I20260809 12:04:10.866678 10186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:10.867683 10202 master.cc:537] Master@127.0.147.124:43561 shutting down...
I20260809 12:04:10.872342 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } } }
I20260809 12:04:10.872628 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:04:10.889405 10381 raft_consensus.cc:858] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } }
W20260809 12:04:10.889652 10381 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } }
I20260809 12:04:10.920430 10381 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 30cfede53d33410e951ee4fbdccb0205 (127.0.147.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:10.928870 10420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b99ca339e2e440ba634712c2a35f0bf. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:10.929253 10420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 12:04:10.928958 10453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:10.929910 10453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 12:04:10.930045 10184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } } }
I20260809 12:04:10.930302 10184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:10.928879 10458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: NON_VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: true } } }
I20260809 12:04:10.930567 10458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260809 12:04:10.931584 9356 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 12:04:10.934092 10202 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 12:04:10.934571 10202 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 12:04:10.934756 10202 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: stopping tablet replica
W20260809 12:04:10.934930 9356 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf -> Peer 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124:43561): Couldn't send request to peer 0305fc037f3e402b83e867dc3bc2b91f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 12:04:10.950855 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 9455
I20260809 12:04:10.960798 10202 master.cc:552] Master@127.0.147.124:43561 shutdown complete.
I20260809 12:04:10.960932 10202 master_runner.cc:305] Clearing existing system tablet
I20260809 12:04:10.961380 10202 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:10.961488 10202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:10.964694 10202 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 12:04:10.966611 10461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:10.967082 10202 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:10.967518 10202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 12:04:10.967703 10202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal
uuid: "0305fc037f3e402b83e867dc3bc2b91f"
format_stamp: "Formatted at 2026-08-09 12:04:10 on dist-test-slave-sb2z"
server_key: "aecd979c3202b91ef93ac78dbec519b7"
server_key_iv: "d13cd9d2acf758d66b83154500561188"
server_key_version: "encryptionkey@0"
I20260809 12:04:10.972620 10202 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:10.980259 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:42359
--local_ip_for_outbound_sockets=127.0.147.65
--tserver_master_addrs=127.0.147.126:35267
--webserver_port=46071
--webserver_interface=127.0.147.65
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab
--principal=oryx/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.147.126:35267,127.0.147.125:35491
--tserver_master_addrs=127.0.147.126:35267,127.0.147.125:35491,127.0.147.124:43561 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:04:10.980504 10202 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 12:04:10.980669 10202 log.cc:1196] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 12:04:10.981235 10202 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Deleting consensus metadata
I20260809 12:04:10.981487 10202 master_runner.cc:315] Copying system tablet from 127.0.147.126:35267
I20260809 12:04:10.988246 10202 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: tablet copy: Beginning tablet copy session from remote peer at address 127.0.147.126:35267
I20260809 12:04:11.004586 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.989122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48454 (local address 127.0.147.126:35267)
0809 12:04:10.989320 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:10.989327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:10.991046 (+ 1719us) server_negotiation.cc:407] Connection header received
0809 12:04:10.991263 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:10.991275 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:10.991356 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:10.991467 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:10.992644 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.993593 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.994908 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.995366 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.995499 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.998016 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:10.998021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:10.998024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:10.998055 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:11.000833 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.001522 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.001528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.001531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.001659 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.002065 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.002070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.002073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.002365 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:11.002539 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:11.003650 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0809 12:04:11.004032 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:04:11.006215 10469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:10.989104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:48454)
0809 12:04:10.990917 (+ 1813us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:10.990936 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:10.991114 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:10.991507 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:10.991512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:10.992284 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:10.992483 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.992492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.993745 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.993750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.994763 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:10.994774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:10.995574 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:10.995583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:10.995784 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:10.996423 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:10.996441 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:10.997810 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:11.000989 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:11.001004 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:11.001008 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:11.001393 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:11.001776 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:11.001781 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:11.001785 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:11.001938 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:11.002743 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:11.002749 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:11.003095 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 12:04:11.005285 (+ 2190us) client_negotiation.cc:240] Negotiation successful
0809 12:04:11.005753 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1678,"thread_start_us":1554,"threads_started":1}
I20260809 12:04:11.006954 9401 tablet_copy_service.cc:140] P 7b99ca339e2e440ba634712c2a35f0bf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0305fc037f3e402b83e867dc3bc2b91f ({username='oryx', principal='oryx/127.0.147.124@KRBTEST.COM'} at 127.0.0.1:48454)
I20260809 12:04:11.007141 9401 tablet_copy_service.cc:161] P 7b99ca339e2e440ba634712c2a35f0bf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0305fc037f3e402b83e867dc3bc2b91f at {username='oryx', principal='oryx/127.0.147.124@KRBTEST.COM'} at 127.0.0.1:48454: session id = 0305fc037f3e402b83e867dc3bc2b91f-00000000000000000000000000000000
I20260809 12:04:11.009516 9401 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 12:04:11.011096 10202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 12:04:11.015806 10202 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: tablet copy: Starting download of 0 data blocks...
I20260809 12:04:11.016086 10202 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: tablet copy: Starting download of 1 WAL segments...
I20260809 12:04:11.018168 10202 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 12:04:11.021888 9401 tablet_copy_service.cc:342] P 7b99ca339e2e440ba634712c2a35f0bf: Request end of tablet copy session 0305fc037f3e402b83e867dc3bc2b91f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.147.124@KRBTEST.COM'} at 127.0.0.1:48454
I20260809 12:04:11.022020 9401 tablet_copy_service.cc:433] P 7b99ca339e2e440ba634712c2a35f0bf: ending tablet copy session 0305fc037f3e402b83e867dc3bc2b91f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0305fc037f3e402b83e867dc3bc2b91f
I20260809 12:04:11.032171 10202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:11.035094 10471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:11.036223 10472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:11.040542 10202 server_base.cc:1034] running on GCE node
W20260809 12:04:11.040822 10475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:11.041322 10202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:11.041983 10202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:11.043202 10202 hybrid_clock.cc:648] HybridClock initialized: now 1786277051043145 us; error 44 us; skew 500 ppm
I20260809 12:04:11.043452 10202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:11 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277051, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:11.045713 10202 init.cc:377] Logged in from keytab as oryx/127.0.147.124@KRBTEST.COM (short username oryx)
I20260809 12:04:11.048444 10202 webserver.cc:460] Webserver started at http://127.0.147.124:39327/ using document root <none> and password file <none>
I20260809 12:04:11.048784 10202 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:11.048882 10202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:11.052652 10202 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 12:04:11.054778 10485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:11.055275 10202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 12:04:11.055400 10202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal
uuid: "0305fc037f3e402b83e867dc3bc2b91f"
format_stamp: "Formatted at 2026-08-09 12:04:10 on dist-test-slave-sb2z"
server_key: "aecd979c3202b91ef93ac78dbec519b7"
server_key_iv: "d13cd9d2acf758d66b83154500561188"
server_key_version: "encryptionkey@0"
I20260809 12:04:11.055620 10202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:11.071604 10202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:11.076139 10202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:11.105993 10202 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.124:43561
I20260809 12:04:11.106035 10556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.124:43561 every 8 connection(s)
I20260809 12:04:11.107048 10202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
I20260809 12:04:11.112676 10557 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:04:11.116468 10557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Bootstrap starting.
I20260809 12:04:11.131610 10557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:04:11.132105 10557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: Bootstrap complete.
I20260809 12:04:11.133268 10557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } }
I20260809 12:04:11.133566 10557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0305fc037f3e402b83e867dc3bc2b91f, State: Initialized, Role: LEARNER
I20260809 12:04:11.133816 10557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } }
I20260809 12:04:11.134677 10560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } } }
I20260809 12:04:11.135051 10557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:11.135179 10560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: This master's current role is: LEARNER
I20260809 12:04:11.144284 10578 catalog_manager.cc:1259] Loaded cluster ID: 6fe03249f99e4815a346c82328311b7a
I20260809 12:04:11.144393 10578 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: loading cluster ID for follower catalog manager: success
I20260809 12:04:11.148104 10578 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: acquiring CA information for follower catalog manager: success
I20260809 12:04:11.150378 10578 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:04:11.384632 10187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } } }
I20260809 12:04:11.385054 10187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:11.399207 10581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:11.385090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:43561 (local address 127.0.0.1:53394)
0809 12:04:11.385625 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:11.385656 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:11.385872 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:11.387892 (+ 2020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:11.387898 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:11.387934 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:11.388356 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:11.388372 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.391310 (+ 2938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.391317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:11.394843 (+ 3526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:11.394871 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.397674 (+ 2803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.397681 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:11.398054 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:11.398094 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 12:04:11.398234 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 12:04:11.398409 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":296,"threads_started":1}
I20260809 12:04:11.399318 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:11.385704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53394 (local address 127.0.147.124:43561)
0809 12:04:11.387301 (+ 1597us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:11.387311 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:11.387348 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:04:11.387452 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:11.387462 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:11.387683 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:11.387841 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:11.388588 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.391107 (+ 2519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.395135 (+ 4028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.397437 (+ 2302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.397642 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:11.398016 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:11.398276 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 12:04:11.398500 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1351,"thread_start_us":1102,"threads_started":1}
I20260809 12:04:11.401482 10521 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260809 12:04:11.402364 10560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b99ca339e2e440ba634712c2a35f0bf. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: NON_VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: true } } }
I20260809 12:04:11.402904 10560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: This master's current role is: LEARNER
I20260809 12:04:11.403198 10187 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: attempting to promote NON_VOTER 0305fc037f3e402b83e867dc3bc2b91f to VOTER
I20260809 12:04:11.404207 10187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } }
I20260809 12:04:11.406643 10521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 LEARNER]: Refusing update from remote peer 7b99ca339e2e440ba634712c2a35f0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 12:04:11.406803 10264 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Refusing update from remote peer 7b99ca339e2e440ba634712c2a35f0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 12:04:11.407502 10186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 12:04:11.407967 10184 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 12:04:11.411057 10184 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:11.414256 10521 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:11.415858 10184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:11.416220 10184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:11.416131 10187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:11.416558 10187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:11.416891 10264 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:11.419780 10560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:11.420038 10560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:04:11.423514 10187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:11.423985 10187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:11.428365 10586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b99ca339e2e440ba634712c2a35f0bf" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:11.428833 10586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 12:04:11.450466 10470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:11.450857 10470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:11.451114 10470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:11.461362 10470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:11.461608 10470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:11.461858 10470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:04:11.475251 10470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab
--principal=oryx/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:42359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=46071
--tserver_master_addrs=127.0.147.126:35267,127.0.147.125:35491,127.0.147.124:43561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:11.476902 10470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:11.478434 10470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:11.498405 10591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:11.499346 10592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:11.499743 10594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:11.500883 10470 server_base.cc:1034] running on GCE node
I20260809 12:04:11.501638 10470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:11.502882 10470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:11.504141 10470 hybrid_clock.cc:648] HybridClock initialized: now 1786277051504063 us; error 60 us; skew 500 ppm
I20260809 12:04:11.504526 10470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:11 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277051, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:11.509140 10470 init.cc:377] Logged in from keytab as oryx/127.0.147.65@KRBTEST.COM (short username oryx)
I20260809 12:04:11.511493 10470 webserver.cc:460] Webserver started at http://127.0.147.65:46071/ using document root <none> and password file <none>
I20260809 12:04:11.512518 10470 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:11.512776 10470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:11.519799 10470 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 12:04:11.523507 10603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:11.525190 10470 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:04:11.525460 10470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "acc522b25d284580a2af45212bc42e40"
format_stamp: "Formatted at 2026-08-09 12:04:08 on dist-test-slave-sb2z"
server_key: "12ebb8ff9ba4f9dcdbac6e305ad82352"
server_key_iv: "e2596bc258c724262aee0b6bf6099d98"
server_key_version: "encryptionkey@0"
I20260809 12:04:11.526978 10470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:11.552743 10470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:11.558858 10470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:11.559294 10470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:11.560842 10470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:11.563028 10470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:11.563268 10470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:11.563467 10470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:11.563642 10470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:11.652276 10470 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:42359
I20260809 12:04:11.652510 10758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:42359 every 8 connection(s)
I20260809 12:04:11.654168 10470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:04:11.658581 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 10470
I20260809 12:04:11.659183 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 9657
Aug 09 12:04:11 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277051, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.124@KRBTEST.COM
Aug 09 12:04:11 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277051, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
Aug 09 12:04:11 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277051, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.125@KRBTEST.COM
Aug 09 12:04:11 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.66@KRBTEST.COM
I20260809 12:04:11.721118 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:11.661125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:48419 (local address 127.0.147.124:43561)
0809 12:04:11.661372 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:11.661382 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:11.685905 (+ 24523us) server_negotiation.cc:407] Connection header received
0809 12:04:11.691188 (+ 5283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:11.691198 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:11.691320 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:11.691410 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:11.695507 (+ 4097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.696942 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.700837 (+ 3895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.701954 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.702112 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:11.709781 (+ 7669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:11.709788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:11.709793 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:11.709853 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:11.715718 (+ 5865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.717098 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.717116 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.717122 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.717458 (+ 336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.718268 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.718274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.718278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.718723 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:11.718850 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:11.720206 (+ 1356us) server_negotiation.cc:299] Negotiation successful
0809 12:04:11.720546 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 12:04:11.722249 10765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:11.658552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:59573 (local address 127.0.147.126:35267)
0809 12:04:11.673325 (+ 14773us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:11.673335 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:11.685867 (+ 12532us) server_negotiation.cc:407] Connection header received
0809 12:04:11.691153 (+ 5286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:11.691162 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:11.691313 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:11.691693 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:11.695660 (+ 3967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.697221 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.700795 (+ 3574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.701867 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.702011 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:11.710710 (+ 8699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:11.710720 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:11.710726 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:11.710765 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:11.716885 (+ 6120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.718171 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.718177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.718180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.718475 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.719266 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.719271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.719275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.719735 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:11.720013 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:11.720884 (+ 871us) server_negotiation.cc:299] Negotiation successful
0809 12:04:11.721460 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12631,"thread_start_us":12736,"threads_started":1}
I20260809 12:04:11.724285 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:11.660678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:43561 (local address 127.0.147.65:48419)
0809 12:04:11.685672 (+ 24994us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:11.685703 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:11.690589 (+ 4886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:11.692143 (+ 1554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:11.692159 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:11.693620 (+ 1461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:11.695238 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:11.695271 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.697186 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.697193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:11.700648 (+ 3455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:11.700663 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.702747 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.702758 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:11.703052 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:11.704233 (+ 1181us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:11.704271 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:11.709374 (+ 5103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:11.715901 (+ 6527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:11.715914 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:11.715932 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:11.716926 (+ 994us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:11.717654 (+ 728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:11.717660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:11.717665 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:11.718089 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:11.719078 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:11.719086 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:11.719646 (+ 560us) client_negotiation.cc:769] Sending connection context
0809 12:04:11.721423 (+ 1777us) client_negotiation.cc:240] Negotiation successful
0809 12:04:11.722240 (+ 817us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24802,"mutex_wait_us":248,"spinlock_wait_cycles":37504,"thread_start_us":22681,"threads_started":1}
I20260809 12:04:11.724324 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:11.677676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.147.65:36987)
0809 12:04:11.686321 (+ 8645us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:11.686344 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:11.690584 (+ 4240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:11.693756 (+ 3172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:11.693762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:11.695304 (+ 1542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:11.696087 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:11.696101 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.698182 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.698188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:11.700876 (+ 2688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:11.700889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.702795 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.702806 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:11.703107 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:11.704305 (+ 1198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:11.704322 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:11.709769 (+ 5447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:11.718723 (+ 8954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:11.718729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:11.718732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:11.719410 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:11.720016 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:11.720022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:11.720028 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:11.720374 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:11.721247 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:11.721251 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:11.721414 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 12:04:11.721597 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 12:04:11.722343 (+ 746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8360,"mutex_wait_us":314,"spinlock_wait_cycles":43264,"thread_start_us":6857,"threads_started":1}
I20260809 12:04:11.725179 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:11.678522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:36987 (local address 127.0.147.125:35491)
0809 12:04:11.693128 (+ 14606us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:11.693134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:11.693155 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:04:11.693218 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:11.693224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:11.693299 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:11.694216 (+ 917us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:11.696269 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.697984 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.701095 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.702213 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.702335 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:11.710143 (+ 7808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:11.710152 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:11.710156 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:11.710216 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:11.718565 (+ 8349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.719636 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.719641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.719644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.719855 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.720557 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.720561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.720564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.721064 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:11.722063 (+ 999us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:11.724258 (+ 2195us) server_negotiation.cc:299] Negotiation successful
0809 12:04:11.724503 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14406,"thread_start_us":15049,"threads_started":1}
I20260809 12:04:11.725525 10769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:11.659608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:59573)
0809 12:04:11.685473 (+ 25865us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:11.685552 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:11.690921 (+ 5369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:11.692143 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:11.692158 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:11.693975 (+ 1817us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:11.695377 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:11.695392 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.697484 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.697490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:11.700578 (+ 3088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:11.700593 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.702572 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.702583 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:11.703012 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:11.704254 (+ 1242us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:11.704280 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:11.710309 (+ 6029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:11.717094 (+ 6785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:11.717113 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:11.717118 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:11.717994 (+ 876us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:11.718690 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:11.718695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:11.718699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:11.719114 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:11.720140 (+ 1026us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:11.720152 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:11.720392 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 12:04:11.722876 (+ 2484us) client_negotiation.cc:240] Negotiation successful
0809 12:04:11.723247 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23664,"mutex_wait_us":184,"spinlock_wait_cycles":49024,"thread_start_us":26924,"threads_started":1}
I20260809 12:04:11.726768 10762 heartbeater.cc:347] Connected to a master server at 127.0.147.125:35491
I20260809 12:04:11.726897 10760 heartbeater.cc:347] Connected to a master server at 127.0.147.124:43561
I20260809 12:04:11.727221 10762 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:11.727221 10760 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:11.727366 10761 heartbeater.cc:347] Connected to a master server at 127.0.147.126:35267
I20260809 12:04:11.727571 10761 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:11.729431 10761 heartbeater.cc:511] Master 127.0.147.126:35267 requested a full tablet report, sending...
I20260809 12:04:11.729583 10760 heartbeater.cc:511] Master 127.0.147.124:43561 requested a full tablet report, sending...
I20260809 12:04:11.729640 10762 heartbeater.cc:511] Master 127.0.147.125:35491 requested a full tablet report, sending...
I20260809 12:04:11.731951 9379 ts_manager.cc:194] Re-registered known tserver with Master: acc522b25d284580a2af45212bc42e40 (127.0.147.65:42359)
I20260809 12:04:11.733059 10253 ts_manager.cc:194] Registered new tserver with Master: acc522b25d284580a2af45212bc42e40 (127.0.147.65:42359)
I20260809 12:04:11.733233 10506 ts_manager.cc:194] Registered new tserver with Master: acc522b25d284580a2af45212bc42e40 (127.0.147.65:42359)
I20260809 12:04:11.733358 9379 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:59573
I20260809 12:04:11.735817 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:11.661338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.0.147.66:46105)
0809 12:04:11.698365 (+ 37027us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:11.698375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:11.699689 (+ 1314us) server_negotiation.cc:407] Connection header received
0809 12:04:11.700064 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:11.700075 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:11.700415 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:11.700707 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:11.705053 (+ 4346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.707448 (+ 2395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.710539 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:11.712280 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:11.712391 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:11.721738 (+ 9347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:11.721765 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:11.721782 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:11.721843 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:11.730116 (+ 8273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.731382 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.731390 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.731393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.731655 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:11.732380 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:11.732400 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:11.732404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:11.732954 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:11.733453 (+ 499us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:11.734822 (+ 1369us) server_negotiation.cc:299] Negotiation successful
0809 12:04:11.735138 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36768,"thread_start_us":36352,"threads_started":1}
I20260809 12:04:11.737325 9835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:11.825695 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 9657
I20260809 12:04:11.847961 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:46105
--local_ip_for_outbound_sockets=127.0.147.66
--tserver_master_addrs=127.0.147.126:35267
--webserver_port=39751
--webserver_interface=127.0.147.66
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab
--principal=oryx/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.147.126:35267,127.0.147.125:35491
--tserver_master_addrs=127.0.147.126:35267,127.0.147.125:35491,127.0.147.124:43561 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:12.200394 10774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:12.200680 10774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:12.200778 10774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:12.208074 10774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:12.208185 10774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:12.208307 10774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:04:12.218173 10774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab
--principal=oryx/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:46105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=39751
--tserver_master_addrs=127.0.147.126:35267,127.0.147.125:35491,127.0.147.124:43561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:12.219408 10774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:12.220731 10774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:12.234596 10788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:12.234678 10783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:12.235147 10791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:12.235827 10774 server_base.cc:1034] running on GCE node
I20260809 12:04:12.236439 10774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:12.237325 10774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:12.238530 10774 hybrid_clock.cc:648] HybridClock initialized: now 1786277052238463 us; error 63 us; skew 500 ppm
I20260809 12:04:12.238962 10774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:12 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277052, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:12.242503 10774 init.cc:377] Logged in from keytab as oryx/127.0.147.66@KRBTEST.COM (short username oryx)
I20260809 12:04:12.244068 10774 webserver.cc:460] Webserver started at http://127.0.147.66:39751/ using document root <none> and password file <none>
I20260809 12:04:12.244736 10774 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:12.244858 10774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:12.250840 10774 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 12:04:12.253760 10800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:12.255252 10774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:12.255422 10774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "8fe60c8c3eef4f60838bb7d81fea5387"
format_stamp: "Formatted at 2026-08-09 12:04:08 on dist-test-slave-sb2z"
server_key: "95641e8f7770f2f5566e0681d268363e"
server_key_iv: "6a4d472ca7cc3d239b1c7b3aa7aa735a"
server_key_version: "encryptionkey@0"
I20260809 12:04:12.256587 10774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:12.276243 10774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:12.280543 10774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:12.280846 10774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:12.281854 10774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:12.283250 10774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:12.283361 10774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:12.283499 10774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:12.283641 10774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 12:04:12.319357 10774 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:46105
I20260809 12:04:12.319445 10948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:46105 every 8 connection(s)
I20260809 12:04:12.320966 10774 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:04:12.324786 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 10774
I20260809 12:04:12.325165 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 9849
Aug 09 12:04:12 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.67@KRBTEST.COM
Aug 09 12:04:12 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277052, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.124@KRBTEST.COM
Aug 09 12:04:12 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277052, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.125@KRBTEST.COM
Aug 09 12:04:12 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277052, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:12.362251 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.326948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.0.147.67:34885)
0809 12:04:12.332327 (+ 5379us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:12.332338 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:12.332907 (+ 569us) server_negotiation.cc:407] Connection header received
0809 12:04:12.333143 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:12.333150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:12.333336 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:12.333663 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:12.335852 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.337555 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.339661 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.340944 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.341119 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:12.349792 (+ 8673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:12.349816 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:12.349833 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:12.349885 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:12.356103 (+ 6218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:12.357356 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:12.357365 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:12.357380 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:12.357631 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:12.358385 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:12.358391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:12.358396 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:12.358999 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:12.359516 (+ 517us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:12.361094 (+ 1578us) server_negotiation.cc:299] Negotiation successful
0809 12:04:12.361458 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5161,"thread_start_us":4977,"threads_started":1}
I20260809 12:04:12.364455 10025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:12.365962 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.324092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:55101 (local address 127.0.147.124:43561)
0809 12:04:12.326209 (+ 2117us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:12.326218 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:12.329800 (+ 3582us) server_negotiation.cc:407] Connection header received
0809 12:04:12.336516 (+ 6716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:12.336526 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:12.336630 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:12.336902 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:12.339893 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.341258 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.345593 (+ 4335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.346723 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.346831 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:12.354248 (+ 7417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:12.354256 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:12.354261 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:12.354312 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:12.360327 (+ 6015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:12.361849 (+ 1522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:12.361857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:12.361860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:12.362150 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:12.363087 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:12.363106 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:12.363110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:12.363718 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:12.363855 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:12.365069 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0809 12:04:12.365298 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1988,"thread_start_us":1855,"threads_started":1}
I20260809 12:04:12.366460 10958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.324047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:37615 (local address 127.0.147.125:35491)
0809 12:04:12.326469 (+ 2422us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:12.326475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:12.337204 (+ 10729us) server_negotiation.cc:407] Connection header received
0809 12:04:12.337393 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:12.337398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:12.337479 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:12.337827 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:12.341795 (+ 3968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.342995 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.345844 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.346799 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.346917 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:12.354717 (+ 7800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:12.354724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:12.354729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:12.354770 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:12.361546 (+ 6776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:12.362902 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:12.362907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:12.362910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:12.363128 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:12.363816 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:12.363821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:12.363825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:12.364456 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:12.364637 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:12.365448 (+ 811us) server_negotiation.cc:299] Negotiation successful
0809 12:04:12.365755 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2302,"thread_start_us":2182,"threads_started":1}
I20260809 12:04:12.368441 10962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.327686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:43561 (local address 127.0.147.66:55101)
0809 12:04:12.329487 (+ 1801us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:12.329538 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:12.336220 (+ 6682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:12.337450 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:12.337462 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:12.338476 (+ 1014us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:12.339720 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:12.339740 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.341607 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.341615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:12.345404 (+ 3789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:12.345418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.347410 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.347420 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:12.348022 (+ 602us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:12.349010 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:12.349033 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:12.353789 (+ 4756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:12.360559 (+ 6770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:12.360570 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:12.360586 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:12.361674 (+ 1088us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:12.362510 (+ 836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:12.362519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:12.362523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:12.362937 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:12.364033 (+ 1096us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:12.364043 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:12.364542 (+ 499us) client_negotiation.cc:769] Sending connection context
0809 12:04:12.366364 (+ 1822us) client_negotiation.cc:240] Negotiation successful
0809 12:04:12.366909 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1504,"mutex_wait_us":113,"spinlock_wait_cycles":16768,"thread_start_us":1240,"threads_started":1}
I20260809 12:04:12.369134 10955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.323917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:40809 (local address 127.0.147.126:35267)
0809 12:04:12.324529 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:12.324538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:12.336251 (+ 11713us) server_negotiation.cc:407] Connection header received
0809 12:04:12.337458 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:12.337465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:12.337552 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:12.337861 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:12.342398 (+ 4537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.344033 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.346949 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.348066 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.348189 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:12.356625 (+ 8436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:12.356632 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:12.356636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:12.356784 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:12.362578 (+ 5794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:12.363867 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:12.363876 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:12.363879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:12.364122 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:12.364698 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:12.364703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:12.364706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:12.365179 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:12.365308 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:12.368188 (+ 2880us) server_negotiation.cc:299] Negotiation successful
0809 12:04:12.368633 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":326,"threads_started":1}
I20260809 12:04:12.369688 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.328544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.147.66:37615)
0809 12:04:12.336986 (+ 8442us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:12.337004 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:12.337217 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:12.338080 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:12.338085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:12.338928 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:12.341597 (+ 2669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:12.341613 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.343234 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.343240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:12.345620 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:12.345637 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.347673 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.347684 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:12.348047 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:12.348912 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:12.348998 (+ 86us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:12.354429 (+ 5431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:12.361751 (+ 7322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:12.361756 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:12.361759 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:12.362710 (+ 951us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:12.363281 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:12.363290 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:12.363294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:12.363672 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:12.364762 (+ 1090us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:12.364766 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:12.365004 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 12:04:12.366769 (+ 1765us) client_negotiation.cc:240] Negotiation successful
0809 12:04:12.367154 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8307,"mutex_wait_us":118,"thread_start_us":4883,"threads_started":1}
I20260809 12:04:12.369776 10963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.328153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.66:40809)
0809 12:04:12.333702 (+ 5549us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:12.333725 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:12.336249 (+ 2524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:12.338041 (+ 1792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:12.338045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:12.339005 (+ 960us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:12.342001 (+ 2996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:12.342016 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.344261 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.344266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:12.346781 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:12.346799 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.348596 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.348605 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:12.348963 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:12.349855 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:12.349876 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:12.356249 (+ 6373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:12.362768 (+ 6519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:12.362772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:12.362776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:12.363717 (+ 941us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:12.364279 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:12.364284 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:12.364288 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:12.364542 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:12.365385 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:12.365390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:12.365600 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 12:04:12.365780 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 12:04:12.366377 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5411,"mutex_wait_us":65,"spinlock_wait_cycles":33536,"thread_start_us":8549,"threads_started":1}
I20260809 12:04:12.370702 10949 heartbeater.cc:347] Connected to a master server at 127.0.147.124:43561
I20260809 12:04:12.371167 10951 heartbeater.cc:347] Connected to a master server at 127.0.147.125:35491
I20260809 12:04:12.371417 10951 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:12.371908 10950 heartbeater.cc:347] Connected to a master server at 127.0.147.126:35267
I20260809 12:04:12.371196 10949 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:12.372812 10950 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:12.373325 10951 heartbeater.cc:511] Master 127.0.147.125:35491 requested a full tablet report, sending...
I20260809 12:04:12.373769 10950 heartbeater.cc:511] Master 127.0.147.126:35267 requested a full tablet report, sending...
I20260809 12:04:12.374643 10949 heartbeater.cc:511] Master 127.0.147.124:43561 requested a full tablet report, sending...
I20260809 12:04:12.376866 10253 ts_manager.cc:194] Registered new tserver with Master: 8fe60c8c3eef4f60838bb7d81fea5387 (127.0.147.66:46105)
I20260809 12:04:12.376966 10506 ts_manager.cc:194] Registered new tserver with Master: 8fe60c8c3eef4f60838bb7d81fea5387 (127.0.147.66:46105)
I20260809 12:04:12.378160 9379 ts_manager.cc:194] Re-registered known tserver with Master: 8fe60c8c3eef4f60838bb7d81fea5387 (127.0.147.66:46105)
I20260809 12:04:12.379245 9379 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:40809
I20260809 12:04:12.468715 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 9849
I20260809 12:04:12.496124 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:34885
--local_ip_for_outbound_sockets=127.0.147.67
--tserver_master_addrs=127.0.147.126:35267
--webserver_port=43277
--webserver_interface=127.0.147.67
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab
--principal=oryx/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.147.126:35267,127.0.147.125:35491
--tserver_master_addrs=127.0.147.126:35267,127.0.147.125:35491,127.0.147.124:43561 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:04:12.736806 10761 heartbeater.cc:503] Master 127.0.147.126:35267 was elected leader, sending a full tablet report...
W20260809 12:04:12.842955 10966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:12.843297 10966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:12.843472 10966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:12.850816 10966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:12.850924 10966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:12.851046 10966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:04:12.861594 10966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab
--principal=oryx/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:34885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=43277
--tserver_master_addrs=127.0.147.126:35267,127.0.147.125:35491,127.0.147.124:43561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:12.862855 10966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:12.864200 10966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:12.878019 10971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:12.878010 10972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:12.878198 10974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:12.879202 10966 server_base.cc:1034] running on GCE node
I20260809 12:04:12.879861 10966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:12.880738 10966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:12.881934 10966 hybrid_clock.cc:648] HybridClock initialized: now 1786277052881880 us; error 52 us; skew 500 ppm
I20260809 12:04:12.882279 10966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:12 dist-test-slave-sb2z krb5kdc[9313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277052, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:12.885419 10966 init.cc:377] Logged in from keytab as oryx/127.0.147.67@KRBTEST.COM (short username oryx)
I20260809 12:04:12.886947 10966 webserver.cc:460] Webserver started at http://127.0.147.67:43277/ using document root <none> and password file <none>
I20260809 12:04:12.887717 10966 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:12.887867 10966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:12.893039 10966 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:04:12.895692 10987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:12.896963 10966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:04:12.897135 10966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "d48348aed937491da6b7e827d3d7295f"
format_stamp: "Formatted at 2026-08-09 12:04:09 on dist-test-slave-sb2z"
server_key: "180d44c13222e790b897e0ac966f2310"
server_key_iv: "139340aec8d210df7a52a4624552bf3f"
server_key_version: "encryptionkey@0"
I20260809 12:04:12.898164 10966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:12.913151 10966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:12.917070 10966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:12.917344 10966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:12.918275 10966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:12.919648 10966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:12.919770 10966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:12.919904 10966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:12.920004 10966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:12.955215 10966 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:34885
I20260809 12:04:12.955264 11142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:34885 every 8 connection(s)
I20260809 12:04:12.956871 10966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:04:12.963879 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 10966
Aug 09 12:04:13 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277052, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.125@KRBTEST.COM
Aug 09 12:04:13 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277052, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.124@KRBTEST.COM
Aug 09 12:04:13 dist-test-slave-sb2z krb5kdc[9313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277052, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:13.029863 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.960168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:58523 (local address 127.0.147.124:43561)
0809 12:04:12.986583 (+ 26415us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:12.986587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:12.987755 (+ 1168us) server_negotiation.cc:407] Connection header received
0809 12:04:12.990361 (+ 2606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:12.990367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:12.990454 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:12.990560 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:12.998715 (+ 8155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.000029 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.006788 (+ 6759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.007696 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.007821 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.017823 (+ 10002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.017833 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.017840 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.017896 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.024961 (+ 7065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.026582 (+ 1621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.026588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.026591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.026741 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.027474 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.027479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.027482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.027898 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.028022 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.029187 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.029440 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26291,"thread_start_us":26155,"threads_started":1}
I20260809 12:04:13.032554 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.960046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:44433 (local address 127.0.147.126:35267)
0809 12:04:12.986143 (+ 26097us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:12.986153 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:12.986182 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:04:12.990361 (+ 4179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:12.990367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:12.990454 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:12.990577 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:13.003961 (+ 13384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.005066 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.014467 (+ 9401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.015114 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.015246 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.024935 (+ 9689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.024943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.024947 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.024996 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.028639 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.029424 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.029429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.029432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.029576 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.030056 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.030061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.030064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.030511 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.030649 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.031350 (+ 701us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.031685 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25903,"thread_start_us":26527,"threads_started":1}
I20260809 12:04:13.032577 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.960362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:53689 (local address 127.0.147.125:35491)
0809 12:04:12.985567 (+ 25205us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:12.985576 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:12.988482 (+ 2906us) server_negotiation.cc:407] Connection header received
0809 12:04:12.990794 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:12.990799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:12.990887 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:12.991097 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:12.993865 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.995366 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.000386 (+ 5020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.001471 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.001612 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.021343 (+ 19731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.021350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.021355 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.021409 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.026219 (+ 4810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.027454 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.027465 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.027472 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.027704 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.028359 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.028367 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.028374 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.028861 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.029097 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.031305 (+ 2208us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.031663 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25024,"thread_start_us":25235,"threads_started":1}
I20260809 12:04:13.034875 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.960752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.67:44433)
0809 12:04:12.983777 (+ 23025us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:12.983862 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:12.990006 (+ 6144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:12.990940 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:12.990953 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:12.997907 (+ 6954us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:13.003797 (+ 5890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.003818 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.012390 (+ 8572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.012398 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.014303 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.014323 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.015743 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.015752 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.015996 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.021350 (+ 5354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:13.021374 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:13.024691 (+ 3317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:13.028815 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:13.028820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:13.028823 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:13.029287 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:13.029724 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:13.029729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:13.029732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:13.029900 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:13.030733 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:13.030739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:13.030935 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 12:04:13.032860 (+ 1925us) client_negotiation.cc:240] Negotiation successful
0809 12:04:13.033361 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22528,"thread_start_us":27050,"threads_started":1}
I20260809 12:04:13.034647 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.960752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.147.67:53689)
0809 12:04:12.988207 (+ 27455us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:12.988231 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:12.990017 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:12.991365 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:12.991371 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:12.992135 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:12.993647 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:12.993664 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:12.995677 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:12.995686 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:12.998454 (+ 2768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:12.998470 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.004112 (+ 5642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.004120 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.004407 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.008904 (+ 4497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:13.008945 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:13.019349 (+ 10404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:13.026453 (+ 7104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:13.026462 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:13.026469 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:13.027247 (+ 778us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:13.027892 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:13.027901 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:13.027908 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:13.028182 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:13.029187 (+ 1005us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:13.029196 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:13.029418 (+ 222us) client_negotiation.cc:769] Sending connection context
0809 12:04:13.029632 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 12:04:13.030340 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27302,"mutex_wait_us":1614,"thread_start_us":24650,"threads_started":1}
I20260809 12:04:13.037003 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.961114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:43561 (local address 127.0.147.67:58523)
0809 12:04:12.987582 (+ 26468us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:12.987608 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:12.990017 (+ 2409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:12.990939 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:12.990952 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:12.992098 (+ 1146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:12.993066 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:12.993111 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.000260 (+ 7149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.000269 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.006580 (+ 6311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.006601 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.008150 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.008158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.008409 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.010851 (+ 2442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:13.010883 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:13.017503 (+ 6620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:13.025644 (+ 8141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:13.025656 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:13.025679 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:13.026422 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:13.026908 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:13.026915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:13.026920 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:13.027299 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:13.028146 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:13.028157 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:13.028645 (+ 488us) client_negotiation.cc:769] Sending connection context
0809 12:04:13.030138 (+ 1493us) client_negotiation.cc:240] Negotiation successful
0809 12:04:13.033023 (+ 2885us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26272,"mutex_wait_us":2591,"spinlock_wait_cycles":26880,"thread_start_us":27388,"threads_started":1}
I20260809 12:04:13.037206 11147 heartbeater.cc:347] Connected to a master server at 127.0.147.125:35491
I20260809 12:04:13.037267 11146 heartbeater.cc:347] Connected to a master server at 127.0.147.126:35267
I20260809 12:04:13.037678 11147 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:13.037696 11146 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:13.039032 11145 heartbeater.cc:347] Connected to a master server at 127.0.147.124:43561
I20260809 12:04:13.039249 11145 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:13.040467 11147 heartbeater.cc:511] Master 127.0.147.125:35491 requested a full tablet report, sending...
I20260809 12:04:13.041832 11146 heartbeater.cc:511] Master 127.0.147.126:35267 requested a full tablet report, sending...
I20260809 12:04:13.042227 11145 heartbeater.cc:511] Master 127.0.147.124:43561 requested a full tablet report, sending...
I20260809 12:04:13.044790 10253 ts_manager.cc:194] Registered new tserver with Master: d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885)
I20260809 12:04:13.045122 10506 ts_manager.cc:194] Registered new tserver with Master: d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885)
I20260809 12:04:13.046897 9379 ts_manager.cc:194] Re-registered known tserver with Master: d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885)
I20260809 12:04:13.048554 9379 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:44433
I20260809 12:04:13.061465 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.991760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.0.147.126:35267)
0809 12:04:13.023277 (+ 31517us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.023284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.023310 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:04:13.023365 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.023369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.023438 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.023603 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:13.035336 (+ 11733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.037895 (+ 2559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.041357 (+ 3462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.042441 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.042623 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.050024 (+ 7401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.050030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.050033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.050064 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.055465 (+ 5401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.056756 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.056762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.056766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.056962 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.058531 (+ 1569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.058538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.058544 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.059055 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.059384 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.060825 (+ 1441us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.061121 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31380,"thread_start_us":32052,"threads_started":1}
I20260809 12:04:13.061939 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.999253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53396 (local address 127.0.147.124:43561)
0809 12:04:13.015275 (+ 16022us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.015281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.022723 (+ 7442us) server_negotiation.cc:407] Connection header received
0809 12:04:13.022790 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.022796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.022911 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.023134 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:13.034497 (+ 11363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.036361 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.041845 (+ 5484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.042672 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.043509 (+ 837us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.051367 (+ 7858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.051375 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.051379 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.051426 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.056821 (+ 5395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.058048 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.058052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.058054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.058238 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.058811 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.058816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.058819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.059470 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.059868 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.060397 (+ 529us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.060746 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15864,"thread_start_us":16146,"threads_started":1}
I20260809 12:04:13.068280 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:12.998541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.0.147.125:35491)
0809 12:04:13.014888 (+ 16347us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.014894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.035294 (+ 20400us) server_negotiation.cc:407] Connection header received
0809 12:04:13.035353 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.035358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.035480 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.035828 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:13.047410 (+ 11582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.049169 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.052926 (+ 3757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.053794 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.053947 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.058976 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.058988 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.058997 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.059037 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.064382 (+ 5345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.065261 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.065269 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.065278 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.065494 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.066032 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.066036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.066038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.066401 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.066726 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.067214 (+ 488us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.067519 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16193,"thread_start_us":19164,"threads_started":1}
I20260809 12:04:13.067296 9379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:04:13.078207 9379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:04:13.142751 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.116087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:42359 (local address 127.0.0.1:45750)
0809 12:04:13.130264 (+ 14177us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:13.130287 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:13.130568 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:13.133296 (+ 2728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:13.133300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:13.133328 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:13.134138 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.134154 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.136363 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.136385 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.138709 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.138722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.141232 (+ 2510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.141244 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.141658 (+ 414us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.141729 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 12:04:13.141831 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:04:13.142016 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13995,"mutex_wait_us":4,"thread_start_us":20660,"threads_started":1}
I20260809 12:04:13.143656 11182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.105101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:46105 (local address 127.0.0.1:38034)
0809 12:04:13.129060 (+ 23959us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:13.129085 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:13.129377 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:13.132321 (+ 2944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:13.132326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:13.132353 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:13.133076 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.133088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.135882 (+ 2794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.135891 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.139672 (+ 3781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.139688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.142548 (+ 2860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.142555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.142750 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.142784 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:04:13.142880 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 12:04:13.143022 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23739,"thread_start_us":24402,"threads_started":1}
I20260809 12:04:13.144275 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.116458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45750 (local address 127.0.147.65:42359)
0809 12:04:13.132637 (+ 16179us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.132648 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.132681 (+ 33us) server_negotiation.cc:407] Connection header received
0809 12:04:13.132781 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.132788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.133064 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.133341 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:13.134528 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.136201 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.138898 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.140828 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.143280 (+ 2452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.143503 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.143681 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.143869 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15913,"thread_start_us":17206,"threads_started":1}
I20260809 12:04:13.145866 11184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.105483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38034 (local address 127.0.147.66:46105)
0809 12:04:13.131312 (+ 25829us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.131321 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.131355 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:04:13.131725 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.131733 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.132089 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.132342 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:13.133266 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.135614 (+ 2348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.139910 (+ 4296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.141971 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.142094 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.142337 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.144046 (+ 1709us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.144274 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25669,"thread_start_us":25561,"threads_started":1}
I20260809 12:04:13.149158 10669 tablet_service.cc:1467] Processing CreateTablet for tablet 7efd1a6a31454ddba112dafab258225c (DEFAULT_TABLE table=test-table [id=440e93911b0a47059fd75bdc76588426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:13.150687 10669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efd1a6a31454ddba112dafab258225c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:13.152608 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.104493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:34885 (local address 127.0.0.1:57460)
0809 12:04:13.127824 (+ 23331us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:13.127850 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:13.138861 (+ 11011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:13.140846 (+ 1985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:13.140851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:13.140888 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:13.141977 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.141991 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.144840 (+ 2849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.144846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.148319 (+ 3473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.148335 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.151338 (+ 3003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.151345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.151644 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.151680 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:04:13.151776 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 12:04:13.151917 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23081,"thread_start_us":26966,"threads_started":1}
I20260809 12:04:13.153710 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.105046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57460 (local address 127.0.147.67:34885)
0809 12:04:13.135395 (+ 30349us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.135413 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.135481 (+ 68us) server_negotiation.cc:407] Connection header received
0809 12:04:13.139731 (+ 4250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.139741 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.140087 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.140389 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:13.142227 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.144661 (+ 2434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.148504 (+ 3843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.150691 (+ 2187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.150831 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.151055 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.152884 (+ 1829us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.153122 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30054,"thread_start_us":30489,"threads_started":1}
I20260809 12:04:13.154116 10856 tablet_service.cc:1467] Processing CreateTablet for tablet 7efd1a6a31454ddba112dafab258225c (DEFAULT_TABLE table=test-table [id=440e93911b0a47059fd75bdc76588426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:13.156352 10856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efd1a6a31454ddba112dafab258225c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:13.160386 11049 tablet_service.cc:1467] Processing CreateTablet for tablet 7efd1a6a31454ddba112dafab258225c (DEFAULT_TABLE table=test-table [id=440e93911b0a47059fd75bdc76588426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:13.162457 11049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efd1a6a31454ddba112dafab258225c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:13.175262 11187 tablet_bootstrap.cc:492] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: Bootstrap starting.
I20260809 12:04:13.177795 11188 tablet_bootstrap.cc:492] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: Bootstrap starting.
I20260809 12:04:13.181437 11187 tablet_bootstrap.cc:654] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:13.183256 11187 log.cc:824] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:13.184020 11188 tablet_bootstrap.cc:654] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:13.186582 11191 tablet_bootstrap.cc:492] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: Bootstrap starting.
I20260809 12:04:13.186640 11188 log.cc:824] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:13.190449 11188 tablet_bootstrap.cc:492] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: No bootstrap required, opened a new log
I20260809 12:04:13.190625 11187 tablet_bootstrap.cc:492] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: No bootstrap required, opened a new log
I20260809 12:04:13.190795 11188 ts_tablet_manager.cc:1397] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260809 12:04:13.191133 11187 ts_tablet_manager.cc:1397] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.003s
I20260809 12:04:13.193558 11191 tablet_bootstrap.cc:654] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:13.195847 11191 log.cc:824] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:13.201433 11191 tablet_bootstrap.cc:492] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: No bootstrap required, opened a new log
I20260809 12:04:13.201949 11191 ts_tablet_manager.cc:1397] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260809 12:04:13.203184 11188 raft_consensus.cc:348] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:13.203787 11188 raft_consensus.cc:374] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:13.203971 11188 raft_consensus.cc:729] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe60c8c3eef4f60838bb7d81fea5387, State: Initialized, Role: FOLLOWER
I20260809 12:04:13.204742 11188 consensus_queue.cc:260] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:13.207504 11188 ts_tablet_manager.cc:1428] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: Time spent starting tablet: real 0.017s user 0.011s sys 0.006s
I20260809 12:04:13.208222 10950 heartbeater.cc:503] Master 127.0.147.126:35267 was elected leader, sending a full tablet report...
I20260809 12:04:13.208503 11187 raft_consensus.cc:348] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:13.209272 11187 raft_consensus.cc:374] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:13.209443 11187 raft_consensus.cc:729] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc522b25d284580a2af45212bc42e40, State: Initialized, Role: FOLLOWER
I20260809 12:04:13.210304 11187 consensus_queue.cc:260] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:13.213778 11187 ts_tablet_manager.cc:1428] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: Time spent starting tablet: real 0.022s user 0.016s sys 0.004s
I20260809 12:04:13.221876 11191 raft_consensus.cc:348] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:13.222534 11191 raft_consensus.cc:374] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:13.222708 11191 raft_consensus.cc:729] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d48348aed937491da6b7e827d3d7295f, State: Initialized, Role: FOLLOWER
I20260809 12:04:13.223333 11191 consensus_queue.cc:260] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:13.225131 11146 heartbeater.cc:503] Master 127.0.147.126:35267 was elected leader, sending a full tablet report...
I20260809 12:04:13.225914 11191 ts_tablet_manager.cc:1428] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: Time spent starting tablet: real 0.024s user 0.020s sys 0.001s
I20260809 12:04:13.237958 11194 raft_consensus.cc:482] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:04:13.238364 11194 raft_consensus.cc:504] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:13.241098 11194 leader_election.cc:288] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885), acc522b25d284580a2af45212bc42e40 (127.0.147.65:42359)
I20260809 12:04:13.254489 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.241620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:48119 (local address 127.0.147.65:42359)
0809 12:04:13.241921 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.241936 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.243942 (+ 2006us) server_negotiation.cc:407] Connection header received
0809 12:04:13.244326 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.244332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.244440 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.244580 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:13.246838 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.248179 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.251572 (+ 3393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.253147 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.253362 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.253469 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.253965 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.254100 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:04:13.254866 11197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.241285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:42359 (local address 127.0.147.66:48119)
0809 12:04:13.243654 (+ 2369us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:13.243686 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:13.244146 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:13.244636 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:13.244644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:13.244674 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:13.246679 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.246691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.248361 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.248367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.251369 (+ 3002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.251384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.253561 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.253573 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.253786 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.253813 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 12:04:13.253940 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 12:04:13.254080 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2129,"mutex_wait_us":1,"thread_start_us":1423,"threads_started":1}
I20260809 12:04:13.256359 10696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efd1a6a31454ddba112dafab258225c" candidate_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc522b25d284580a2af45212bc42e40" is_pre_election: true
I20260809 12:04:13.257069 10696 raft_consensus.cc:2454] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe60c8c3eef4f60838bb7d81fea5387 in term 0.
I20260809 12:04:13.258106 10805 leader_election.cc:302] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe60c8c3eef4f60838bb7d81fea5387, acc522b25d284580a2af45212bc42e40; no voters:
I20260809 12:04:13.258754 11194 raft_consensus.cc:2781] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:04:13.259035 11194 raft_consensus.cc:482] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:04:13.259251 11194 raft_consensus.cc:3037] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:13.259428 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.241512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:45563 (local address 127.0.147.67:34885)
0809 12:04:13.241731 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.241738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.244447 (+ 2709us) server_negotiation.cc:407] Connection header received
0809 12:04:13.244719 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.244725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.244831 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.244940 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:13.250980 (+ 6040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.252418 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.255851 (+ 3433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.257773 (+ 1922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.257945 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.258016 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.258872 (+ 856us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.259061 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:04:13.259496 11199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.242043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:34885 (local address 127.0.147.66:45563)
0809 12:04:13.244168 (+ 2125us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:13.244185 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:13.244535 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:13.245216 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:13.245221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:13.245255 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:13.246422 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.246438 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.252607 (+ 6169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.252613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.255632 (+ 3019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:13.255648 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.258343 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.258351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:13.258659 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.258697 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:04:13.258788 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 12:04:13.258938 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1958,"thread_start_us":1827,"threads_started":1}
I20260809 12:04:13.261157 11075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efd1a6a31454ddba112dafab258225c" candidate_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48348aed937491da6b7e827d3d7295f" is_pre_election: true
I20260809 12:04:13.261974 11075 raft_consensus.cc:2454] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe60c8c3eef4f60838bb7d81fea5387 in term 0.
I20260809 12:04:13.264420 11194 raft_consensus.cc:504] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:13.265432 11194 leader_election.cc:288] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [CANDIDATE]: Term 1 election: Requested vote from peers d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885), acc522b25d284580a2af45212bc42e40 (127.0.147.65:42359)
I20260809 12:04:13.265926 11075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efd1a6a31454ddba112dafab258225c" candidate_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48348aed937491da6b7e827d3d7295f"
I20260809 12:04:13.265991 10696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efd1a6a31454ddba112dafab258225c" candidate_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc522b25d284580a2af45212bc42e40"
I20260809 12:04:13.266240 11075 raft_consensus.cc:3037] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:13.266301 10696 raft_consensus.cc:3037] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:13.269627 10696 raft_consensus.cc:2454] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe60c8c3eef4f60838bb7d81fea5387 in term 1.
I20260809 12:04:13.269671 11075 raft_consensus.cc:2454] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe60c8c3eef4f60838bb7d81fea5387 in term 1.
I20260809 12:04:13.270155 10805 leader_election.cc:302] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe60c8c3eef4f60838bb7d81fea5387, acc522b25d284580a2af45212bc42e40; no voters:
I20260809 12:04:13.270522 11194 raft_consensus.cc:2781] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:13.271405 11194 raft_consensus.cc:686] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe60c8c3eef4f60838bb7d81fea5387, State: Running, Role: LEADER
I20260809 12:04:13.272192 11194 consensus_queue.cc:237] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:13.278743 9378 catalog_manager.cc:5495] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe60c8c3eef4f60838bb7d81fea5387 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } health_report { overall_health: UNKNOWN } } }
W20260809 12:04:13.334342 10952 tablet.cc:2369] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:04:13.337236 11184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.329343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.0.147.66:46105)
0809 12:04:13.329626 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.329632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.329653 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:04:13.329877 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.329883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.329981 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.330170 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:04:13.330680 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.331934 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.333990 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.334746 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.334943 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.335765 (+ 822us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:04:13.336423 (+ 658us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:04:13.336607 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.336663 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.336748 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:04:13.347882 10696 raft_consensus.cc:1261] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [term 1 FOLLOWER]: Refusing update from remote peer 8fe60c8c3eef4f60838bb7d81fea5387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:13.347893 11075 raft_consensus.cc:1261] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [term 1 FOLLOWER]: Refusing update from remote peer 8fe60c8c3eef4f60838bb7d81fea5387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:13.349275 11200 consensus_queue.cc:1035] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:13.349860 11194 consensus_queue.cc:1035] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:13.369836 11203 mvcc.cc:205] Tried to move back new op lower bound from 7316590810500644864 to 7316590810208038912. Current Snapshot: MvccSnapshot[applied={T|T < 7316590810500644864}]
I20260809 12:04:13.373518 11205 mvcc.cc:205] Tried to move back new op lower bound from 7316590810500644864 to 7316590810208038912. Current Snapshot: MvccSnapshot[applied={T|T < 7316590810500644864}]
I20260809 12:04:13.376492 11204 mvcc.cc:205] Tried to move back new op lower bound from 7316590810500644864 to 7316590810208038912. Current Snapshot: MvccSnapshot[applied={T|T < 7316590810500644864}]
I20260809 12:04:13.399475 9378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:13.400013 9378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:13.407809 9378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b99ca339e2e440ba634712c2a35f0bf: Sending DeleteTablet for 3 replicas of tablet 7efd1a6a31454ddba112dafab258225c
I20260809 12:04:13.409410 11049 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efd1a6a31454ddba112dafab258225c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:13 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:57460
I20260809 12:04:13.411008 10669 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efd1a6a31454ddba112dafab258225c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:13 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:45750
I20260809 12:04:13.415076 11221 tablet_replica.cc:330] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: stopping tablet replica
I20260809 12:04:13.415743 11221 raft_consensus.cc:2229] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:13.417029 11221 raft_consensus.cc:2258] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:13.417929 10856 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efd1a6a31454ddba112dafab258225c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:13 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:38034
I20260809 12:04:13.420094 11221 ts_tablet_manager.cc:1905] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:13.426559 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 9327
I20260809 12:04:13.428545 9414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:13.431984 11223 tablet_replica.cc:330] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: stopping tablet replica
I20260809 12:04:13.432775 11223 raft_consensus.cc:2229] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:04:13.434140 11223 raft_consensus.cc:2258] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:13.436004 11218 tablet_replica.cc:330] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: stopping tablet replica
I20260809 12:04:13.436640 11218 raft_consensus.cc:2229] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:13.437395 11223 ts_tablet_manager.cc:1905] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:13.437606 11218 raft_consensus.cc:2258] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:13.440274 11218 ts_tablet_manager.cc:1905] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:13.440513 11221 ts_tablet_manager.cc:1918] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:13.440848 11221 log.cc:1196] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/7efd1a6a31454ddba112dafab258225c
I20260809 12:04:13.442142 11221 ts_tablet_manager.cc:1939] T 7efd1a6a31454ddba112dafab258225c P acc522b25d284580a2af45212bc42e40: Deleting consensus metadata
I20260809 12:04:13.448370 11223 ts_tablet_manager.cc:1918] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:13.448745 11223 log.cc:1196] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/7efd1a6a31454ddba112dafab258225c
I20260809 12:04:13.449786 11223 ts_tablet_manager.cc:1939] T 7efd1a6a31454ddba112dafab258225c P 8fe60c8c3eef4f60838bb7d81fea5387: Deleting consensus metadata
I20260809 12:04:13.454887 11218 ts_tablet_manager.cc:1918] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:13.455228 11218 log.cc:1196] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/7efd1a6a31454ddba112dafab258225c
I20260809 12:04:13.460669 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.414123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48348 (local address 127.0.147.125:35491)
0809 12:04:13.414395 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.414402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.424823 (+ 10421us) server_negotiation.cc:407] Connection header received
0809 12:04:13.425108 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.425114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.425215 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.425323 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:13.435736 (+ 10413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.437451 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.441217 (+ 3766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.442557 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.442725 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.449533 (+ 6808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.449543 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.449550 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.449602 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.455291 (+ 5689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.456828 (+ 1537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.456835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.456839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.457086 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.457766 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.457772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.457775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.458320 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.458625 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.459592 (+ 967us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.460007 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:04:13.464788 11218 ts_tablet_manager.cc:1939] T 7efd1a6a31454ddba112dafab258225c P d48348aed937491da6b7e827d3d7295f: Deleting consensus metadata
I20260809 12:04:13.488426 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.425346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.0.147.124:43561)
0809 12:04:13.425607 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.425624 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.463709 (+ 38085us) server_negotiation.cc:407] Connection header received
0809 12:04:13.464069 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.464078 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.464191 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.464283 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:13.467267 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.468729 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.471247 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.472402 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.472553 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.478599 (+ 6046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.478607 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.478613 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.478659 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.484215 (+ 5556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.485373 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.485379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.485384 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.485605 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.486198 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.486204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.486208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.486713 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.487077 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.487590 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.487877 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 12:04:13.597375 9356 catalog_manager.cc:4841] TS 8fe60c8c3eef4f60838bb7d81fea5387 (127.0.147.66:46105): tablet 7efd1a6a31454ddba112dafab258225c (table test-table [id=440e93911b0a47059fd75bdc76588426]) successfully deleted
I20260809 12:04:13.597930 9357 catalog_manager.cc:4841] TS acc522b25d284580a2af45212bc42e40 (127.0.147.65:42359): tablet 7efd1a6a31454ddba112dafab258225c (table test-table [id=440e93911b0a47059fd75bdc76588426]) successfully deleted
I20260809 12:04:13.598356 9354 catalog_manager.cc:4841] TS d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885): tablet 7efd1a6a31454ddba112dafab258225c (table test-table [id=440e93911b0a47059fd75bdc76588426]) successfully deleted
I20260809 12:04:13.598598 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 9327
I20260809 12:04:13.727895 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.676536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.0.147.124:43561)
0809 12:04:13.677105 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.677124 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.687485 (+ 10361us) server_negotiation.cc:407] Connection header received
0809 12:04:13.688506 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.688526 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.688973 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.689380 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:13.699826 (+ 10446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.702127 (+ 2301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.708375 (+ 6248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.709447 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.709824 (+ 377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.715736 (+ 5912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.715747 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.715750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.715798 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.721159 (+ 5361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.722632 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.722648 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.722654 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.722970 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.724033 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.724053 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.724058 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.724663 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.725215 (+ 552us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.726100 (+ 885us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.727133 (+ 1033us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260809 12:04:13.730659 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:13.681783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.0.147.125:35491)
0809 12:04:13.682402 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:13.682419 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:13.687504 (+ 5085us) server_negotiation.cc:407] Connection header received
0809 12:04:13.690675 (+ 3171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:13.690692 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:13.690879 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:13.691255 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:13.699307 (+ 8052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.702823 (+ 3516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.710304 (+ 7481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:13.712046 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:13.712268 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:13.718210 (+ 5942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:13.718220 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:13.718225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:13.718288 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:13.724113 (+ 5825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.725264 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.725276 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.725287 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.725674 (+ 387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:13.726665 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:13.726774 (+ 109us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:13.726778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:13.727840 (+ 1062us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:13.728300 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:13.729158 (+ 858us) server_negotiation.cc:299] Negotiation successful
0809 12:04:13.729601 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260809 12:04:14.381053 11235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.379678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.66:32819)
0809 12:04:14.380495 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.380741 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":360,"threads_started":1}
W20260809 12:04:14.382036 10950 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:35267 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
I20260809 12:04:14.386962 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.385780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:53589)
0809 12:04:14.386384 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.386622 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":278,"threads_started":1}
W20260809 12:04:14.387725 10761 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:35267 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
I20260809 12:04:14.390524 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.389559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.67:60869)
0809 12:04:14.390116 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.390280 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":288,"threads_started":1}
W20260809 12:04:14.390959 11146 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:35267 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
I20260809 12:04:14.908921 11240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7b99ca339e2e440ba634712c2a35f0bf)
I20260809 12:04:14.908977 11241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7b99ca339e2e440ba634712c2a35f0bf)
I20260809 12:04:14.909392 11240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } }
I20260809 12:04:14.909453 11241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } }
I20260809 12:04:14.911588 11240 leader_election.cc:288] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267), 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124:43561)
I20260809 12:04:14.916132 11241 leader_election.cc:288] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267), 30cfede53d33410e951ee4fbdccb0205 (127.0.147.125:35491)
I20260809 12:04:14.917042 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.911786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:39788)
0809 12:04:14.916109 (+ 4323us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.916488 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4055,"threads_started":1}
I20260809 12:04:14.918301 11243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.911808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:39790)
0809 12:04:14.917330 (+ 5522us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.917862 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5238,"thread_start_us":5017,"threads_started":1}
W20260809 12:04:14.918941 10231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
W20260809 12:04:14.919260 10487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
I20260809 12:04:14.925935 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.925324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:39794)
0809 12:04:14.925556 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.925720 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 12:04:14.926504 10487 leader_election.cc:334] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267): Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
I20260809 12:04:14.929067 11243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.928462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:39804)
0809 12:04:14.928691 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.928814 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260809 12:04:14.929540 10231 leader_election.cc:334] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267): Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
I20260809 12:04:14.931823 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.912624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:35491 (local address 127.0.0.1:48376)
0809 12:04:14.917687 (+ 5063us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.917719 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:14.917946 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:14.919864 (+ 1918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:14.919876 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:14.919944 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:14.920795 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:14.920824 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:14.924317 (+ 3493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:14.924333 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:14.928015 (+ 3682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:14.928049 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:14.930284 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:14.930295 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:14.930701 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:14.930803 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 12:04:14.930983 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 12:04:14.931176 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4852,"mutex_wait_us":114,"thread_start_us":2945,"threads_started":1}
I20260809 12:04:14.931957 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.912097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.0.147.125:35491)
0809 12:04:14.914833 (+ 2736us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:14.914856 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:14.918139 (+ 3283us) server_negotiation.cc:407] Connection header received
0809 12:04:14.918275 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:14.918295 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:14.918442 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:14.921080 (+ 2638us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:14.921679 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:14.924073 (+ 2394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:14.928325 (+ 4252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:14.930014 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:14.930233 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:14.930484 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:14.931030 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 12:04:14.931342 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2422,"thread_start_us":2109,"threads_started":1}
I20260809 12:04:14.933351 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.912810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:43561 (local address 127.0.0.1:53424)
0809 12:04:14.918352 (+ 5542us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.918391 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:14.918624 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:14.920449 (+ 1825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:14.920468 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:14.920531 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:14.921319 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:14.921345 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:14.925775 (+ 4430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:14.925790 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:14.929023 (+ 3233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:14.929040 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:14.931305 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:14.931320 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:14.931697 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:14.931805 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 12:04:14.931948 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 12:04:14.932166 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5261,"mutex_wait_us":1,"thread_start_us":4721,"threads_started":1}
I20260809 12:04:14.934383 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.917747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53424 (local address 127.0.147.124:43561)
0809 12:04:14.919347 (+ 1600us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:14.919360 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:14.919390 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:04:14.919486 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:14.919492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:14.919637 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:14.920050 (+ 413us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:14.922100 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:14.925405 (+ 3305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:14.929241 (+ 3836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:14.930906 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:14.932334 (+ 1428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:14.932625 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:14.933323 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 12:04:14.933624 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1414,"server-negotiator.queue_time_us":1396,"spinlock_wait_cycles":2304,"thread_start_us":1165,"threads_started":1}
I20260809 12:04:14.935364 10264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0305fc037f3e402b83e867dc3bc2b91f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "30cfede53d33410e951ee4fbdccb0205" is_pre_election: true
I20260809 12:04:14.935626 10521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30cfede53d33410e951ee4fbdccb0205" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0305fc037f3e402b83e867dc3bc2b91f" is_pre_election: true
I20260809 12:04:14.936030 10264 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0305fc037f3e402b83e867dc3bc2b91f in term 1.
I20260809 12:04:14.936241 10521 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30cfede53d33410e951ee4fbdccb0205 in term 1.
I20260809 12:04:14.936805 10489 leader_election.cc:302] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0305fc037f3e402b83e867dc3bc2b91f, 30cfede53d33410e951ee4fbdccb0205; no voters: 7b99ca339e2e440ba634712c2a35f0bf
I20260809 12:04:14.936982 10232 leader_election.cc:302] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0305fc037f3e402b83e867dc3bc2b91f, 30cfede53d33410e951ee4fbdccb0205; no voters: 7b99ca339e2e440ba634712c2a35f0bf
I20260809 12:04:14.937397 11241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 12:04:14.937541 11240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 12:04:14.937633 11241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7b99ca339e2e440ba634712c2a35f0bf)
I20260809 12:04:14.937829 11241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:04:14.937798 11240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7b99ca339e2e440ba634712c2a35f0bf)
I20260809 12:04:14.937978 11240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:04:14.941324 11240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } }
I20260809 12:04:14.941401 11241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } }
I20260809 12:04:14.942154 11240 leader_election.cc:288] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 2 election: Requested vote from peers 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267), 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124:43561)
I20260809 12:04:14.942981 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.942297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:39832)
0809 12:04:14.942604 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.942727 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260809 12:04:14.942919 10264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0305fc037f3e402b83e867dc3bc2b91f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "30cfede53d33410e951ee4fbdccb0205"
I20260809 12:04:14.943342 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.942630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:39836)
0809 12:04:14.942868 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.943015 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 12:04:14.943265 10521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30cfede53d33410e951ee4fbdccb0205" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0305fc037f3e402b83e867dc3bc2b91f"
I20260809 12:04:14.943454 10264 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0305fc037f3e402b83e867dc3bc2b91f in current term 2: Already voted for candidate 30cfede53d33410e951ee4fbdccb0205 in this term.
I20260809 12:04:14.943719 11241 leader_election.cc:288] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [CANDIDATE]: Term 2 election: Requested vote from peers 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267), 30cfede53d33410e951ee4fbdccb0205 (127.0.147.125:35491)
I20260809 12:04:14.943789 10521 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 30cfede53d33410e951ee4fbdccb0205 in current term 2: Already voted for candidate 0305fc037f3e402b83e867dc3bc2b91f in this term.
I20260809 12:04:14.945315 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.944688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:39862)
0809 12:04:14.944936 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.945088 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 12:04:14.945317 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:14.944592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:39852)
0809 12:04:14.944829 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:14.945011 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260809 12:04:14.945626 10231 leader_election.cc:334] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267): Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
W20260809 12:04:14.945727 10487 leader_election.cc:334] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267): Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
I20260809 12:04:14.945771 10231 leader_election.cc:302] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30cfede53d33410e951ee4fbdccb0205; no voters: 0305fc037f3e402b83e867dc3bc2b91f, 7b99ca339e2e440ba634712c2a35f0bf
I20260809 12:04:14.945976 10487 leader_election.cc:302] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0305fc037f3e402b83e867dc3bc2b91f; no voters: 30cfede53d33410e951ee4fbdccb0205, 7b99ca339e2e440ba634712c2a35f0bf
I20260809 12:04:14.946374 11240 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 12:04:14.946487 11241 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 12:04:15.384456 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:15.383087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.66:50153)
0809 12:04:15.384046 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:15.384200 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":695,"threads_started":1}
I20260809 12:04:15.390875 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:15.388848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:49693)
0809 12:04:15.390122 (+ 1274us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:15.390489 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1017,"thread_start_us":763,"threads_started":1}
I20260809 12:04:15.393919 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:15.392604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.67:58915)
0809 12:04:15.393387 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:15.393605 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":420,"threads_started":1}
I20260809 12:04:16.386179 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.385448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.66:43953)
0809 12:04:16.385896 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.385980 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":233,"threads_started":1}
I20260809 12:04:16.393149 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.392307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:58103)
0809 12:04:16.392793 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.392914 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":219,"threads_started":1}
I20260809 12:04:16.395440 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.394846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.67:34489)
0809 12:04:16.395215 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.395290 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":196,"threads_started":1}
I20260809 12:04:16.481957 11264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:04:16.482219 11264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } }
I20260809 12:04:16.482939 11264 leader_election.cc:288] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267), 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124:43561)
I20260809 12:04:16.483628 10521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30cfede53d33410e951ee4fbdccb0205" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0305fc037f3e402b83e867dc3bc2b91f" is_pre_election: true
I20260809 12:04:16.483901 10521 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30cfede53d33410e951ee4fbdccb0205 in term 2.
I20260809 12:04:16.484194 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.483131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:40086)
0809 12:04:16.483753 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.483924 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":304,"threads_started":1}
I20260809 12:04:16.484402 10232 leader_election.cc:302] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0305fc037f3e402b83e867dc3bc2b91f, 30cfede53d33410e951ee4fbdccb0205; no voters:
I20260809 12:04:16.484768 11264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 12:04:16.484927 11264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:04:16.485087 11264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 2 FOLLOWER]: Advancing to term 3
I20260809 12:04:16.485785 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.485402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:40094)
0809 12:04:16.485575 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.485653 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 12:04:16.485966 10231 leader_election.cc:334] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267): Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
I20260809 12:04:16.487432 11264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } }
I20260809 12:04:16.488037 11264 leader_election.cc:288] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 3 election: Requested vote from peers 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267), 0305fc037f3e402b83e867dc3bc2b91f (127.0.147.124:43561)
I20260809 12:04:16.488533 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.488168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:40102)
0809 12:04:16.488340 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.488412 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 12:04:16.488529 10521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30cfede53d33410e951ee4fbdccb0205" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0305fc037f3e402b83e867dc3bc2b91f"
I20260809 12:04:16.488770 10521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 2 FOLLOWER]: Advancing to term 3
I20260809 12:04:16.489408 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.489101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:40118)
0809 12:04:16.489242 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.489314 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:04:16.489553 10231 leader_election.cc:334] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267): Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
I20260809 12:04:16.491146 10521 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30cfede53d33410e951ee4fbdccb0205 in term 3.
I20260809 12:04:16.491601 10232 leader_election.cc:302] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0305fc037f3e402b83e867dc3bc2b91f, 30cfede53d33410e951ee4fbdccb0205; no voters: 7b99ca339e2e440ba634712c2a35f0bf
I20260809 12:04:16.491892 11264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 12:04:16.492285 11264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [term 3 LEADER]: Becoming Leader. State: Replica: 30cfede53d33410e951ee4fbdccb0205, State: Running, Role: LEADER
I20260809 12:04:16.492672 11264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } }
I20260809 12:04:16.494388 11268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30cfede53d33410e951ee4fbdccb0205. Latest consensus state: current_term: 3 leader_uuid: "30cfede53d33410e951ee4fbdccb0205" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:16.494632 11268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:16.495292 11271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:16.499356 11271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:16.500293 11271 catalog_manager.cc:1259] Loaded cluster ID: 6fe03249f99e4815a346c82328311b7a
I20260809 12:04:16.500399 11271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:16.501735 11271 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:16.502682 11271 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Loaded TSK: 0
I20260809 12:04:16.503566 11271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:04:16.529286 10253 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:04:16.537719 10253 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:04:16.542371 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.541884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:40138)
0809 12:04:16.542085 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.542184 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 12:04:16.542395 10521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [term 3 FOLLOWER]: Refusing update from remote peer 30cfede53d33410e951ee4fbdccb0205: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 12:04:16.543126 11268 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 12:04:16.543620 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.543150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:40150)
0809 12:04:16.543333 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.543432 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 12:04:16.543854 10231 consensus_peers.cc:487] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 -> Peer 7b99ca339e2e440ba634712c2a35f0bf (127.0.147.126:35267): Couldn't send request to peer 7b99ca339e2e440ba634712c2a35f0bf. Status: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 12:04:16.545997 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30cfede53d33410e951ee4fbdccb0205. Latest consensus state: current_term: 3 leader_uuid: "30cfede53d33410e951ee4fbdccb0205" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:16.546236 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:04:16.548202 11268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "30cfede53d33410e951ee4fbdccb0205" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:16.548561 11268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:16.549337 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "30cfede53d33410e951ee4fbdccb0205" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7b99ca339e2e440ba634712c2a35f0bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 35267 } } peers { permanent_uuid: "30cfede53d33410e951ee4fbdccb0205" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 35491 } attrs { promote: false } } peers { permanent_uuid: "0305fc037f3e402b83e867dc3bc2b91f" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 43561 } attrs { promote: false } } }
I20260809 12:04:16.549619 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0305fc037f3e402b83e867dc3bc2b91f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:04:16.574018 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.556931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57472 (local address 127.0.147.67:34885)
0809 12:04:16.568297 (+ 11366us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:16.568302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:16.568322 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:04:16.568393 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:16.568397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:16.568470 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:16.568771 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:16.569346 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.570181 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.571972 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.572903 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.573057 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.573214 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:16.573576 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 12:04:16.573713 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11244,"thread_start_us":11099,"threads_started":1}
I20260809 12:04:16.574013 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.556797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:34885 (local address 127.0.0.1:57472)
0809 12:04:16.557101 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.557121 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:16.557526 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:16.568792 (+ 11266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:16.568797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:16.568830 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:16.569177 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.569187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.570314 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.570321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.571814 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.571827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.573208 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.573216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.573397 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.573423 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 12:04:16.573528 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 12:04:16.573672 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":3}
I20260809 12:04:16.575052 11049 tablet_service.cc:1467] Processing CreateTablet for tablet 32448db1bd62431795f34af52230bc54 (DEFAULT_TABLE table=test-table [id=665d37f31c78462b88cc963432ccde36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:16.575722 11049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32448db1bd62431795f34af52230bc54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:16.581997 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.557093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:46105 (local address 127.0.0.1:38052)
0809 12:04:16.568955 (+ 11862us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.568981 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:16.569285 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:16.572622 (+ 3337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:16.572630 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:16.572665 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:16.574651 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.574659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.576203 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.576212 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.579011 (+ 2799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.579030 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.580841 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.580847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.581114 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.581175 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 12:04:16.581270 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 12:04:16.581462 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11657,"mutex_wait_us":1406,"threads_started":1}
I20260809 12:04:16.583570 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.557819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.0.147.66:46105)
0809 12:04:16.571961 (+ 14142us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:16.571971 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:16.571996 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:04:16.572085 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:16.572093 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:16.572189 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:16.572307 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:16.574874 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.576042 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.579232 (+ 3190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.580599 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.582585 (+ 1986us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.582775 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:16.582846 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 12:04:16.583033 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13939,"thread_start_us":15401,"threads_started":1}
I20260809 12:04:16.584498 11286 tablet_bootstrap.cc:492] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: Bootstrap starting.
I20260809 12:04:16.586000 10856 tablet_service.cc:1467] Processing CreateTablet for tablet 32448db1bd62431795f34af52230bc54 (DEFAULT_TABLE table=test-table [id=665d37f31c78462b88cc963432ccde36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:16.586151 11286 tablet_bootstrap.cc:654] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:16.586498 10856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32448db1bd62431795f34af52230bc54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:16.588165 11286 tablet_bootstrap.cc:492] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: No bootstrap required, opened a new log
I20260809 12:04:16.588312 11286 ts_tablet_manager.cc:1397] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 12:04:16.589267 11286 raft_consensus.cc:348] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:16.589476 11286 raft_consensus.cc:374] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:16.589597 11286 raft_consensus.cc:729] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d48348aed937491da6b7e827d3d7295f, State: Initialized, Role: FOLLOWER
I20260809 12:04:16.589967 11286 consensus_queue.cc:260] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:16.590993 11286 ts_tablet_manager.cc:1428] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:16.592358 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.591761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.67:53525)
0809 12:04:16.592045 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.592166 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260809 12:04:16.593475 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.557914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:42359 (local address 127.0.0.1:45762)
0809 12:04:16.574431 (+ 16517us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.574452 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:16.574540 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:16.580507 (+ 5967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:16.580515 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:16.580550 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:16.580773 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.580791 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.583456 (+ 2665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.583461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.585238 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.585251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.592679 (+ 7428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.592687 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.592910 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.592951 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 12:04:16.593028 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 12:04:16.593164 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16411,"thread_start_us":9840,"threads_started":1}
I20260809 12:04:16.594930 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.558499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45762 (local address 127.0.147.65:42359)
0809 12:04:16.580056 (+ 21557us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:16.580075 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:16.580101 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:04:16.580189 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:16.580197 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:16.580313 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:16.580972 (+ 659us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:16.581242 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.582869 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.591242 (+ 8373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.592473 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.594198 (+ 1725us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.594308 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:16.594361 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 12:04:16.594492 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21379,"thread_start_us":21251,"threads_started":1}
I20260809 12:04:16.595891 10669 tablet_service.cc:1467] Processing CreateTablet for tablet 32448db1bd62431795f34af52230bc54 (DEFAULT_TABLE table=test-table [id=665d37f31c78462b88cc963432ccde36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:16.596118 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.595671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.67:60349)
0809 12:04:16.595868 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.595953 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 12:04:16.596379 10669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32448db1bd62431795f34af52230bc54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:16.597118 11147 heartbeater.cc:503] Master 127.0.147.125:35491 was elected leader, sending a full tablet report...
I20260809 12:04:16.601173 11290 tablet_bootstrap.cc:492] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: Bootstrap starting.
I20260809 12:04:16.602591 11291 tablet_bootstrap.cc:492] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: Bootstrap starting.
I20260809 12:04:16.603263 11290 tablet_bootstrap.cc:654] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:16.604271 11291 tablet_bootstrap.cc:654] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:16.605581 11290 tablet_bootstrap.cc:492] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: No bootstrap required, opened a new log
I20260809 12:04:16.605746 11290 ts_tablet_manager.cc:1397] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 12:04:16.606318 11291 tablet_bootstrap.cc:492] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: No bootstrap required, opened a new log
I20260809 12:04:16.606462 11291 ts_tablet_manager.cc:1397] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 12:04:16.606492 11290 raft_consensus.cc:348] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:16.606740 11290 raft_consensus.cc:374] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:16.606841 11290 raft_consensus.cc:729] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe60c8c3eef4f60838bb7d81fea5387, State: Initialized, Role: FOLLOWER
I20260809 12:04:16.607108 11290 consensus_queue.cc:260] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:16.607331 11291 raft_consensus.cc:348] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:16.607611 11291 raft_consensus.cc:374] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:16.607736 11291 raft_consensus.cc:729] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc522b25d284580a2af45212bc42e40, State: Initialized, Role: FOLLOWER
I20260809 12:04:16.608570 11291 consensus_queue.cc:260] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:16.608923 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.608350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.66:51777)
0809 12:04:16.608532 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.608639 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 12:04:16.609921 11290 ts_tablet_manager.cc:1428] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 12:04:16.610777 11291 ts_tablet_manager.cc:1428] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 12:04:16.610805 10951 heartbeater.cc:503] Master 127.0.147.125:35491 was elected leader, sending a full tablet report...
I20260809 12:04:16.611510 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.610588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.66:52033)
0809 12:04:16.611257 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.611348 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570}
I20260809 12:04:16.611972 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.611439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:40841)
0809 12:04:16.611686 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.611783 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260809 12:04:16.613675 10762 heartbeater.cc:503] Master 127.0.147.125:35491 was elected leader, sending a full tablet report...
I20260809 12:04:16.613886 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.613390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:46257)
0809 12:04:16.613572 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.613650 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 12:04:16.682771 10764 tablet.cc:2369] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:04:16.713099 11149 tablet.cc:2369] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:04:16.758384 11296 raft_consensus.cc:482] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:04:16.758756 11296 raft_consensus.cc:504] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:16.760560 11296 leader_election.cc:288] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885), 8fe60c8c3eef4f60838bb7d81fea5387 (127.0.147.66:46105)
I20260809 12:04:16.767467 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.761137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:51675 (local address 127.0.147.66:46105)
0809 12:04:16.761372 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:16.761381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:16.761409 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:04:16.761511 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:16.761516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:16.761597 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:16.761705 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:16.762397 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.763289 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.765305 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.766564 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.766679 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.766732 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:16.767187 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 12:04:16.767276 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:04:16.767735 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.760870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:46105 (local address 127.0.147.65:51675)
0809 12:04:16.761178 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.761204 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:16.761376 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:16.761742 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:16.761747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:16.761774 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:16.762225 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.762240 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.763421 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.763426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.765146 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.765159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.766753 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.766760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.767017 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.767056 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:04:16.767158 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:04:16.767302 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":1}
I20260809 12:04:16.768697 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.760885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:34885 (local address 127.0.147.65:51171)
0809 12:04:16.761807 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.761825 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:16.762121 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:16.762481 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:16.762485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:16.762519 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:16.762832 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.762840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.764238 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.764242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.766110 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:16.766127 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.767937 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.767944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:16.768158 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.768197 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:04:16.768304 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 12:04:16.768392 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"mutex_wait_us":66,"thread_start_us":477,"threads_started":1}
I20260809 12:04:16.768777 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.760872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:51171 (local address 127.0.147.67:34885)
0809 12:04:16.761196 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:16.761204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:16.762067 (+ 863us) server_negotiation.cc:407] Connection header received
0809 12:04:16.762257 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:16.762261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:16.762355 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:16.762454 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:16.763053 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.764109 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.766277 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.767494 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.767686 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.767764 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:16.768346 (+ 582us) server_negotiation.cc:299] Negotiation successful
0809 12:04:16.768486 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:04:16.768687 10885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32448db1bd62431795f34af52230bc54" candidate_uuid: "acc522b25d284580a2af45212bc42e40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" is_pre_election: true
I20260809 12:04:16.769238 10885 raft_consensus.cc:2454] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc522b25d284580a2af45212bc42e40 in term 0.
I20260809 12:04:16.769604 11075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32448db1bd62431795f34af52230bc54" candidate_uuid: "acc522b25d284580a2af45212bc42e40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48348aed937491da6b7e827d3d7295f" is_pre_election: true
I20260809 12:04:16.769860 11075 raft_consensus.cc:2454] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc522b25d284580a2af45212bc42e40 in term 0.
I20260809 12:04:16.769925 10608 leader_election.cc:302] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe60c8c3eef4f60838bb7d81fea5387, acc522b25d284580a2af45212bc42e40; no voters:
I20260809 12:04:16.770411 11296 raft_consensus.cc:2781] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:04:16.770524 11296 raft_consensus.cc:482] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:04:16.770684 11296 raft_consensus.cc:3037] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:16.773239 11296 raft_consensus.cc:504] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:16.774003 11296 leader_election.cc:288] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [CANDIDATE]: Term 1 election: Requested vote from peers d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885), 8fe60c8c3eef4f60838bb7d81fea5387 (127.0.147.66:46105)
I20260809 12:04:16.774395 11075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32448db1bd62431795f34af52230bc54" candidate_uuid: "acc522b25d284580a2af45212bc42e40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48348aed937491da6b7e827d3d7295f"
I20260809 12:04:16.774609 11075 raft_consensus.cc:3037] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:16.774653 10885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32448db1bd62431795f34af52230bc54" candidate_uuid: "acc522b25d284580a2af45212bc42e40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe60c8c3eef4f60838bb7d81fea5387"
I20260809 12:04:16.774838 10885 raft_consensus.cc:3037] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:16.777805 11075 raft_consensus.cc:2454] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc522b25d284580a2af45212bc42e40 in term 1.
I20260809 12:04:16.778126 10885 raft_consensus.cc:2454] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc522b25d284580a2af45212bc42e40 in term 1.
I20260809 12:04:16.778237 10605 leader_election.cc:302] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acc522b25d284580a2af45212bc42e40, d48348aed937491da6b7e827d3d7295f; no voters:
I20260809 12:04:16.778589 11296 raft_consensus.cc:2781] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:16.779093 11296 raft_consensus.cc:686] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 1 LEADER]: Becoming Leader. State: Replica: acc522b25d284580a2af45212bc42e40, State: Running, Role: LEADER
I20260809 12:04:16.779556 11296 consensus_queue.cc:237] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } }
I20260809 12:04:16.780166 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.779769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:60943)
0809 12:04:16.779941 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.780057 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 12:04:16.782611 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.782137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:42875)
0809 12:04:16.782342 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.782431 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 12:04:16.783515 10251 catalog_manager.cc:5495] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 reported cstate change: term changed from 0 to 1, leader changed from <none> to acc522b25d284580a2af45212bc42e40 (127.0.147.65). New cstate: current_term: 1 leader_uuid: "acc522b25d284580a2af45212bc42e40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acc522b25d284580a2af45212bc42e40" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42359 } health_report { overall_health: HEALTHY } } }
I20260809 12:04:16.832809 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.827436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45772 (local address 127.0.147.65:42359)
0809 12:04:16.827638 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:16.827642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:16.827659 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:04:16.827811 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:16.827814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:16.827907 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:16.828085 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:04:16.828436 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.829139 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.830463 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.830917 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.831035 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.831434 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:04:16.832228 (+ 794us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:04:16.832335 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:16.832440 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 12:04:16.832504 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:04:16.837127 11075 raft_consensus.cc:1261] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [term 1 FOLLOWER]: Refusing update from remote peer acc522b25d284580a2af45212bc42e40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:16.837499 10885 raft_consensus.cc:1261] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [term 1 FOLLOWER]: Refusing update from remote peer acc522b25d284580a2af45212bc42e40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:16.837900 11298 consensus_queue.cc:1035] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d48348aed937491da6b7e827d3d7295f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:16.838311 11296 consensus_queue.cc:1035] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe60c8c3eef4f60838bb7d81fea5387" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:16.841344 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.840684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.66:54035)
0809 12:04:16.840953 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.841081 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260809 12:04:16.841614 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.840893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.67:53349)
0809 12:04:16.841273 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.841408 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260809 12:04:16.848260 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.847577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.66:45049)
0809 12:04:16.847884 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.848029 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260809 12:04:16.852401 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.851921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:55285)
0809 12:04:16.852143 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.852245 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 12:04:16.860306 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.859759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.65:54711)
0809 12:04:16.860034 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.860139 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260809 12:04:16.862927 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.861948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.147.67:44827)
0809 12:04:16.862588 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.862710 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499}
I20260809 12:04:16.872792 10251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:16.873123 10251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:16.878898 10251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30cfede53d33410e951ee4fbdccb0205: Sending DeleteTablet for 3 replicas of tablet 32448db1bd62431795f34af52230bc54
I20260809 12:04:16.879879 11049 tablet_service.cc:1514] Processing DeleteTablet for tablet 32448db1bd62431795f34af52230bc54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:16 UTC) from {username='oryx', principal='oryx/127.0.147.125@KRBTEST.COM'} at 127.0.0.1:57472
I20260809 12:04:16.879930 10856 tablet_service.cc:1514] Processing DeleteTablet for tablet 32448db1bd62431795f34af52230bc54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:16 UTC) from {username='oryx', principal='oryx/127.0.147.125@KRBTEST.COM'} at 127.0.0.1:38052
I20260809 12:04:16.880180 10669 tablet_service.cc:1514] Processing DeleteTablet for tablet 32448db1bd62431795f34af52230bc54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:16 UTC) from {username='oryx', principal='oryx/127.0.147.125@KRBTEST.COM'} at 127.0.0.1:45762
I20260809 12:04:16.882467 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 10470
I20260809 12:04:16.895697 11313 tablet_replica.cc:330] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: stopping tablet replica
I20260809 12:04:16.895920 11313 raft_consensus.cc:2229] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:16.895989 11315 tablet_replica.cc:330] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: stopping tablet replica
I20260809 12:04:16.896201 11315 raft_consensus.cc:2229] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:04:16.896262 11313 raft_consensus.cc:2258] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:16.896807 11315 raft_consensus.cc:2258] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:16.898553 11313 ts_tablet_manager.cc:1905] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:16.900413 11315 ts_tablet_manager.cc:1905] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:16.905388 11313 ts_tablet_manager.cc:1918] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:16.905529 11313 log.cc:1196] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/32448db1bd62431795f34af52230bc54
I20260809 12:04:16.906026 11313 ts_tablet_manager.cc:1939] T 32448db1bd62431795f34af52230bc54 P 8fe60c8c3eef4f60838bb7d81fea5387: Deleting consensus metadata
I20260809 12:04:16.907274 10232 catalog_manager.cc:4841] TS 8fe60c8c3eef4f60838bb7d81fea5387 (127.0.147.66:46105): tablet 32448db1bd62431795f34af52230bc54 (table test-table [id=665d37f31c78462b88cc963432ccde36]) successfully deleted
I20260809 12:04:16.907997 11312 tablet_replica.cc:330] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: stopping tablet replica
I20260809 12:04:16.908210 11312 raft_consensus.cc:2229] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:16.908499 11312 raft_consensus.cc:2258] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:16.909575 11312 ts_tablet_manager.cc:1905] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:16.910002 11315 ts_tablet_manager.cc:1918] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:16.910168 11315 log.cc:1196] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/32448db1bd62431795f34af52230bc54
I20260809 12:04:16.910686 11315 ts_tablet_manager.cc:1939] T 32448db1bd62431795f34af52230bc54 P acc522b25d284580a2af45212bc42e40: Deleting consensus metadata
I20260809 12:04:16.911856 10233 catalog_manager.cc:4841] TS acc522b25d284580a2af45212bc42e40 (127.0.147.65:42359): tablet 32448db1bd62431795f34af52230bc54 (table test-table [id=665d37f31c78462b88cc963432ccde36]) successfully deleted
I20260809 12:04:16.916472 11312 ts_tablet_manager.cc:1918] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:16.916663 11312 log.cc:1196] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/32448db1bd62431795f34af52230bc54
I20260809 12:04:16.917201 11312 ts_tablet_manager.cc:1939] T 32448db1bd62431795f34af52230bc54 P d48348aed937491da6b7e827d3d7295f: Deleting consensus metadata
I20260809 12:04:16.918313 10230 catalog_manager.cc:4841] TS d48348aed937491da6b7e827d3d7295f (127.0.147.67:34885): tablet 32448db1bd62431795f34af52230bc54 (table test-table [id=665d37f31c78462b88cc963432ccde36]) successfully deleted
I20260809 12:04:16.920710 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.883913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45784 (local address 127.0.147.65:42359)
0809 12:04:16.884147 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:16.884156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:16.890884 (+ 6728us) server_negotiation.cc:407] Connection header received
0809 12:04:16.891850 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:16.891858 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:16.891995 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:16.892089 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:16.897106 (+ 5017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.898368 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.899839 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:16.900481 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:16.900625 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:16.912886 (+ 12261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:16.912902 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:16.912923 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:16.912964 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:16.917595 (+ 4631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:16.918471 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:16.918480 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:16.918483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:16.918704 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:16.919140 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:16.919143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:16.919145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:16.919626 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:16.919867 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:16.920199 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 12:04:16.920367 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:04:16.921502 10753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:16.979761 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.979307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:40166)
0809 12:04:16.979486 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.979610 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 12:04:16.980805 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:16.980493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:35267 (local address 127.0.0.1:40168)
0809 12:04:16.980632 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:16.980703 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 12:04:17.029328 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 10470
I20260809 12:04:17.081359 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 10774
I20260809 12:04:17.094846 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:17.082184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38066 (local address 127.0.147.66:46105)
0809 12:04:17.082359 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:17.082363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:17.082381 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:04:17.082516 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:17.082520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:17.082585 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:17.082670 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:17.084102 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:17.084826 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:17.086034 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:17.086432 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:17.086529 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:17.089402 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:17.089419 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:17.089434 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:17.089467 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:17.092445 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:17.093114 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:17.093121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:17.093123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:17.093252 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:17.093570 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:17.093573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:17.093575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:17.093914 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:17.094187 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:17.094405 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 12:04:17.094575 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:04:17.095800 10944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:17.219363 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 10774
I20260809 12:04:17.264302 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 10966
I20260809 12:04:17.277402 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:17.265027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57478 (local address 127.0.147.67:34885)
0809 12:04:17.265198 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:17.265203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:17.265221 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:04:17.265349 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:17.265353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:17.265417 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:17.265516 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:17.266805 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:17.267657 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:17.268920 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:17.269320 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:17.269410 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:17.272144 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:17.272158 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:17.272174 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:17.272209 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:17.275091 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:17.275748 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:17.275754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:17.275757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:17.275888 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:17.276204 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:17.276207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:17.276209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:17.276549 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:17.276781 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:17.276987 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 12:04:17.277148 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:04:17.278083 11137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:17.380292 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 10966
I20260809 12:04:17.412060 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 10043
I20260809 12:04:17.413287 10288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:17.543004 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 10043
I20260809 12:04:17.573792 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 10202
I20260809 12:04:17.574972 10548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:17.697009 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 10202
2026-08-09T12:04:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12287 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:04:17 dist-test-slave-sb2z krb5kdc[11331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:04:17 dist-test-slave-sb2z krb5kdc[11331](info): set up 2 sockets
Aug 09 12:04:17 dist-test-slave-sb2z krb5kdc[11331](info): commencing operation
krb5kdc: starting...
W20260809 12:04:19.138511 589 mini_kdc.cc:121] Time spent starting KDC: real 1.382s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:04:19 dist-test-slave-sb2z krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:04:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:04:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.126.keytab.
Entry for principal oryx/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.126.keytab.
I20260809 12:04:19.420097 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:40243
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.126.keytab
--principal=oryx/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:42535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:40243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:19.760492 11345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:19.760776 11345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:19.760975 11345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:19.768437 11345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:19.768540 11345 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 12:04:19.768630 11345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:19.768734 11345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:19.768821 11345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:19.768915 11345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:19.778966 11345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:42535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:40243
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.126.keytab
--principal=oryx/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:40243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:19.780431 11345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:19.781841 11345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:19.794512 11352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:19.794595 11353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:19.794555 11358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:19.795079 11345 server_base.cc:1034] running on GCE node
I20260809 12:04:19.796126 11345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:19.797551 11345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:19.798857 11345 hybrid_clock.cc:648] HybridClock initialized: now 1786277059798795 us; error 61 us; skew 500 ppm
I20260809 12:04:19.799252 11345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:19 dist-test-slave-sb2z krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277059, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:19.802512 11345 init.cc:377] Logged in from keytab as oryx/127.0.147.126@KRBTEST.COM (short username oryx)
I20260809 12:04:19.804086 11345 webserver.cc:460] Webserver started at http://127.0.147.126:40333/ using document root <none> and password file <none>
I20260809 12:04:19.804795 11345 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:19.804934 11345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:19.805279 11345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:19.808286 11345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "72c1f68eabc346e79a299e1f238ab1b3"
format_stamp: "Formatted at 2026-08-09 12:04:19 on dist-test-slave-sb2z"
server_key: "30cc63c2e4097742e565f812466cb5f7"
server_key_iv: "1f26b20315fb6aba336782c821bf574e"
server_key_version: "encryptionkey@0"
I20260809 12:04:19.809105 11345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "72c1f68eabc346e79a299e1f238ab1b3"
format_stamp: "Formatted at 2026-08-09 12:04:19 on dist-test-slave-sb2z"
server_key: "30cc63c2e4097742e565f812466cb5f7"
server_key_iv: "1f26b20315fb6aba336782c821bf574e"
server_key_version: "encryptionkey@0"
I20260809 12:04:19.814284 11345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:04:19.817793 11366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:19.819118 11345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:19.819305 11345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "72c1f68eabc346e79a299e1f238ab1b3"
format_stamp: "Formatted at 2026-08-09 12:04:19 on dist-test-slave-sb2z"
server_key: "30cc63c2e4097742e565f812466cb5f7"
server_key_iv: "1f26b20315fb6aba336782c821bf574e"
server_key_version: "encryptionkey@0"
I20260809 12:04:19.819558 11345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:19.854452 11345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:19.858973 11345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:19.859274 11345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:19.881290 11345 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:40243
I20260809 12:04:19.881314 11450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:40243 every 8 connection(s)
I20260809 12:04:19.882688 11345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:04:19.886322 11451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:19.887259 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 11345
I20260809 12:04:19.887485 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:04:19.888177 589 external_mini_cluster.cc:1428] Setting key 1ae649e8ce235d68cf4fd2386c469fdd
I20260809 12:04:19.900612 11451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3: Bootstrap starting.
Aug 09 12:04:19 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:19.904826 11451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:19.906047 11451 log.cc:824] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:19.909690 11451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3: No bootstrap required, opened a new log
I20260809 12:04:19.913712 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:19.890273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57792 (local address 127.0.147.126:40243)
0809 12:04:19.891167 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:19.891179 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:19.891227 (+ 48us) server_negotiation.cc:407] Connection header received
0809 12:04:19.892889 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:19.892915 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:19.893291 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:19.893610 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:19.895383 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:19.896762 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:19.898570 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:19.899253 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:19.899379 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:19.904699 (+ 5320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:19.904707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:19.904725 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:19.904765 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:19.908831 (+ 4066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:19.909513 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:19.909522 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:19.909530 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:19.909798 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:19.910231 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:19.910240 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:19.910245 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:19.911052 (+ 807us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:19.911366 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:19.911727 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 12:04:19.912246 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":642,"thread_start_us":399,"threads_started":1}
I20260809 12:04:19.920019 11451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c1f68eabc346e79a299e1f238ab1b3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40243 } }
I20260809 12:04:19.920351 11451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:19.920439 11451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c1f68eabc346e79a299e1f238ab1b3, State: Initialized, Role: FOLLOWER
I20260809 12:04:19.921044 11451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c1f68eabc346e79a299e1f238ab1b3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40243 } }
I20260809 12:04:19.921271 11451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:04:19.921378 11451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:04:19.921528 11451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:19.924111 11451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c1f68eabc346e79a299e1f238ab1b3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40243 } }
I20260809 12:04:19.924700 11451 leader_election.cc:302] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72c1f68eabc346e79a299e1f238ab1b3; no voters:
I20260809 12:04:19.925283 11451 leader_election.cc:288] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:04:19.925441 11459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:19.927304 11459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [term 1 LEADER]: Becoming Leader. State: Replica: 72c1f68eabc346e79a299e1f238ab1b3, State: Running, Role: LEADER
I20260809 12:04:19.927968 11451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:19.927850 11459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c1f68eabc346e79a299e1f238ab1b3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40243 } }
I20260809 12:04:19.931231 11460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72c1f68eabc346e79a299e1f238ab1b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c1f68eabc346e79a299e1f238ab1b3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40243 } } }
I20260809 12:04:19.931280 11461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72c1f68eabc346e79a299e1f238ab1b3. Latest consensus state: current_term: 1 leader_uuid: "72c1f68eabc346e79a299e1f238ab1b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c1f68eabc346e79a299e1f238ab1b3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40243 } } }
I20260809 12:04:19.931675 11460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:19.931736 11461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:19.933847 11469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:19.938598 11469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:19.946754 11469 catalog_manager.cc:1347] Generated new cluster ID: a1ecd7f1a447423780ba217566080a30
I20260809 12:04:19.946869 11469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:19.966259 11469 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:04:19.968339 11469 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:19.976171 11469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3: Generated new TSK 0
I20260809 12:04:19.977312 11469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab.
Entry for principal oryx/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab.
I20260809 12:04:20.043044 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40243
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab
--principal=oryx/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:42535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:20.385314 11489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:20.385629 11489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:20.385823 11489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:20.393187 11489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:20.393349 11489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:20.393496 11489 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:04:20.393635 11489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:04:20.403132 11489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:42535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab
--principal=oryx/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.147.126:40243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:20.404433 11489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:20.405773 11489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:20.419298 11495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:20.419231 11496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:20.419338 11501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:20.420163 11489 server_base.cc:1034] running on GCE node
I20260809 12:04:20.420831 11489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:20.421784 11489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:20.423048 11489 hybrid_clock.cc:648] HybridClock initialized: now 1786277060422989 us; error 28 us; skew 500 ppm
I20260809 12:04:20.423384 11489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:20 dist-test-slave-sb2z krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277060, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:20.426748 11489 init.cc:377] Logged in from keytab as oryx/127.0.147.65@KRBTEST.COM (short username oryx)
I20260809 12:04:20.428218 11489 webserver.cc:460] Webserver started at http://127.0.147.65:46243/ using document root <none> and password file <none>
I20260809 12:04:20.428901 11489 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:20.428998 11489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:20.429333 11489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:20.432389 11489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "5bd8d9fdd6fd4792b96d58d568301083"
format_stamp: "Formatted at 2026-08-09 12:04:20 on dist-test-slave-sb2z"
server_key: "92cbde1ef3803562437353c7a5d14ba4"
server_key_iv: "9f68870b17c58d2236d211106f04b410"
server_key_version: "encryptionkey@0"
I20260809 12:04:20.433246 11489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "5bd8d9fdd6fd4792b96d58d568301083"
format_stamp: "Formatted at 2026-08-09 12:04:20 on dist-test-slave-sb2z"
server_key: "92cbde1ef3803562437353c7a5d14ba4"
server_key_iv: "9f68870b17c58d2236d211106f04b410"
server_key_version: "encryptionkey@0"
I20260809 12:04:20.438361 11489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 12:04:20.441665 11509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:20.442898 11489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:20.443085 11489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "5bd8d9fdd6fd4792b96d58d568301083"
format_stamp: "Formatted at 2026-08-09 12:04:20 on dist-test-slave-sb2z"
server_key: "92cbde1ef3803562437353c7a5d14ba4"
server_key_iv: "9f68870b17c58d2236d211106f04b410"
server_key_version: "encryptionkey@0"
I20260809 12:04:20.443298 11489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:20.468358 11489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:20.472812 11489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:20.473136 11489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:20.475860 11489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:20.475978 11489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:20.476125 11489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:20.476497 11489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 09 12:04:20 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277060, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:20.500137 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:20.476509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:52735 (local address 127.0.147.126:40243)
0809 12:04:20.477004 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:20.477012 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:20.478209 (+ 1197us) server_negotiation.cc:407] Connection header received
0809 12:04:20.480481 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:20.480488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:20.480578 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:20.480703 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:20.482995 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:20.484105 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:20.486741 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:20.487373 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:20.487510 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:20.492098 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:20.492102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:20.492106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:20.492151 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:20.495875 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:20.496795 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:20.496801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:20.496804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:20.496971 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:20.497578 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:20.497589 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:20.497591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:20.498005 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:20.498187 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:20.499692 (+ 1505us) server_negotiation.cc:299] Negotiation successful
0809 12:04:20.499874 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":262,"threads_started":1}
I20260809 12:04:20.501366 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:20.477234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40243 (local address 127.0.147.65:52735)
0809 12:04:20.478028 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:20.478081 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:20.480129 (+ 2048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:20.481005 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:20.481022 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:20.481758 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:20.482785 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:20.482824 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:20.484266 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:20.484273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:20.486578 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:20.486599 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:20.487630 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:20.487641 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:20.487859 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:20.488579 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:20.488601 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:20.491850 (+ 3249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:20.496032 (+ 4182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:20.496044 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:20.496071 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:20.496634 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:20.497135 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:20.497143 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:20.497151 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:20.497436 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:20.498159 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:20.498167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:20.498699 (+ 532us) client_negotiation.cc:769] Sending connection context
0809 12:04:20.499110 (+ 411us) client_negotiation.cc:240] Negotiation successful
0809 12:04:20.499685 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":304,"threads_started":1}
I20260809 12:04:20.525568 11676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:38117 every 8 connection(s)
I20260809 12:04:20.525851 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:20.507866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40243 (local address 127.0.147.65:44003)
0809 12:04:20.508856 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:20.508880 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:20.509027 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:20.509543 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:20.509547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:20.510287 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:20.510777 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:20.510794 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:20.512246 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:20.512252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:20.514099 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:20.514117 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:20.515287 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:20.515294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:20.515641 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:20.516473 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:20.516492 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:20.517384 (+ 892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:20.522137 (+ 4753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:20.522146 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:20.522155 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:20.522876 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:20.523463 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:20.523468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:20.523471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:20.523664 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:20.524256 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:20.524264 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:20.524543 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 12:04:20.524744 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 12:04:20.525178 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":464,"threads_started":1}
I20260809 12:04:20.525552 11489 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:38117
I20260809 12:04:20.526345 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:20.507648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:44003 (local address 127.0.147.126:40243)
0809 12:04:20.507877 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:20.507885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:20.508953 (+ 1068us) server_negotiation.cc:407] Connection header received
0809 12:04:20.509203 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:20.509210 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:20.509296 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:20.509399 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:20.510985 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:20.512103 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:20.514291 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:20.514856 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:20.515010 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:20.517752 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:20.517761 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:20.517769 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:20.517825 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:20.521934 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:20.523008 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:20.523016 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:20.523022 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:20.523240 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:20.523765 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:20.523768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:20.523770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:20.524113 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:20.524292 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:20.525722 (+ 1430us) server_negotiation.cc:299] Negotiation successful
0809 12:04:20.525978 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:04:20.527572 11489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:04:20.530066 11677 heartbeater.cc:347] Connected to a master server at 127.0.147.126:40243
I20260809 12:04:20.530328 11677 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:20.530977 11677 heartbeater.cc:511] Master 127.0.147.126:40243 requested a full tablet report, sending...
I20260809 12:04:20.531629 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 11489
I20260809 12:04:20.531838 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:04:20.532582 589 external_mini_cluster.cc:1428] Setting key b8e1f434d9aa1f48695979ed8ffb618e
I20260809 12:04:20.533001 11392 ts_manager.cc:194] Registered new tserver with Master: 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117)
I20260809 12:04:20.534859 11392 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:52735
WARNING: no policy specified for oryx/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab.
Entry for principal oryx/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab.
I20260809 12:04:20.590958 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40243
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab
--principal=oryx/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:42535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:20.938983 11685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:20.939289 11685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:20.939486 11685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:20.946622 11685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:20.946761 11685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:20.946920 11685 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:04:20.947058 11685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:04:20.956681 11685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:42535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab
--principal=oryx/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.147.126:40243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:20.957966 11685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:20.959311 11685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:20.972185 11692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:20.972306 11694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:20.973270 11697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:20.973541 11685 server_base.cc:1034] running on GCE node
I20260809 12:04:20.974174 11685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:20.974990 11685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:20.976306 11685 hybrid_clock.cc:648] HybridClock initialized: now 1786277060976207 us; error 36 us; skew 500 ppm
I20260809 12:04:20.976653 11685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:20 dist-test-slave-sb2z krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277060, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:20.983286 11685 init.cc:377] Logged in from keytab as oryx/127.0.147.66@KRBTEST.COM (short username oryx)
I20260809 12:04:20.984887 11685 webserver.cc:460] Webserver started at http://127.0.147.66:33149/ using document root <none> and password file <none>
I20260809 12:04:20.985605 11685 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:20.985723 11685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:20.986095 11685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:20.989108 11685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "f9d109cc3d55413c8fc4573f3db36192"
format_stamp: "Formatted at 2026-08-09 12:04:20 on dist-test-slave-sb2z"
server_key: "99444fd4fd51e63771d0682a63ce73b0"
server_key_iv: "29c538a4a364ba208bb44f8133d1f807"
server_key_version: "encryptionkey@0"
I20260809 12:04:20.989928 11685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "f9d109cc3d55413c8fc4573f3db36192"
format_stamp: "Formatted at 2026-08-09 12:04:20 on dist-test-slave-sb2z"
server_key: "99444fd4fd51e63771d0682a63ce73b0"
server_key_iv: "29c538a4a364ba208bb44f8133d1f807"
server_key_version: "encryptionkey@0"
I20260809 12:04:20.994975 11685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:04:20.998494 11707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:20.999825 11685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:04:21.000034 11685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "f9d109cc3d55413c8fc4573f3db36192"
format_stamp: "Formatted at 2026-08-09 12:04:20 on dist-test-slave-sb2z"
server_key: "99444fd4fd51e63771d0682a63ce73b0"
server_key_iv: "29c538a4a364ba208bb44f8133d1f807"
server_key_version: "encryptionkey@0"
I20260809 12:04:21.000262 11685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:21.025543 11685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:21.030217 11685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:21.030503 11685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:21.033623 11685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:21.033726 11685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:21.033881 11685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:21.033983 11685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:04:21 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277060, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:21.057739 11719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:21.034255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:37043 (local address 127.0.147.126:40243)
0809 12:04:21.035065 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:21.035069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:21.036071 (+ 1002us) server_negotiation.cc:407] Connection header received
0809 12:04:21.038200 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:21.038207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:21.038303 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:21.038426 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:21.040755 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.041940 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.044739 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.045419 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.045589 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:21.050017 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:21.050022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:21.050025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:21.050058 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:21.053477 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:21.054637 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:21.054644 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:21.054649 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:21.054819 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:21.055486 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:21.055493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:21.055498 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:21.055928 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:21.056122 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:21.057133 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0809 12:04:21.057354 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":559,"threads_started":1}
I20260809 12:04:21.058756 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:21.035000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40243 (local address 127.0.147.66:37043)
0809 12:04:21.035903 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:21.035950 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:21.037909 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:21.038706 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:21.038714 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:21.039490 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:21.040539 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:21.040573 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.042124 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.042136 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:21.044500 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:21.044527 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.045797 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.045808 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:21.046130 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:21.047060 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:21.047090 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:21.049784 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:21.053670 (+ 3886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:21.053677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:21.053688 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:21.054495 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:21.054975 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:21.054980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:21.054984 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:21.055305 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:21.056081 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:21.056121 (+ 40us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:21.056701 (+ 580us) client_negotiation.cc:769] Sending connection context
0809 12:04:21.057033 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 12:04:21.057560 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":367,"threads_started":1}
I20260809 12:04:21.080520 11719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:21.064723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:58445 (local address 127.0.147.126:40243)
0809 12:04:21.064901 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:21.064948 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:21.065676 (+ 728us) server_negotiation.cc:407] Connection header received
0809 12:04:21.065887 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:21.065892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:21.065968 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:21.066069 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:21.067028 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.067999 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.069867 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.070519 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.070652 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:21.073092 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:21.073100 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:21.073110 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:21.073140 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:21.075816 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:21.076774 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:21.076779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:21.076783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:21.076939 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:21.077445 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:21.077452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:21.077457 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:21.077919 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:21.078076 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:21.079767 (+ 1691us) server_negotiation.cc:299] Negotiation successful
0809 12:04:21.080164 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:04:21.079504 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:21.064830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40243 (local address 127.0.147.66:58445)
0809 12:04:21.065551 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:21.065572 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:21.065757 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:21.066095 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:21.066099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:21.066579 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:21.066873 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:21.066882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.068140 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.068145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:21.069612 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:21.069638 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.070733 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.070740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:21.070943 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:21.071696 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:21.071716 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:21.072877 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:21.075961 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:21.075966 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:21.075970 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:21.076599 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:21.077099 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:21.077104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:21.077110 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:21.077302 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:21.078135 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:21.078138 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:21.078384 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 12:04:21.078597 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 12:04:21.078921 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":416,"threads_started":1}
I20260809 12:04:21.093990 11685 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:38159
I20260809 12:04:21.094025 11874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:38159 every 8 connection(s)
I20260809 12:04:21.095201 11685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:04:21.097369 11875 heartbeater.cc:347] Connected to a master server at 127.0.147.126:40243
I20260809 12:04:21.097563 11875 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:21.098166 11875 heartbeater.cc:511] Master 127.0.147.126:40243 requested a full tablet report, sending...
I20260809 12:04:21.098565 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 11685
I20260809 12:04:21.098757 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:04:21.099422 589 external_mini_cluster.cc:1428] Setting key b36e65fed77bcc1d5bfa420049e4599a
I20260809 12:04:21.099720 11392 ts_manager.cc:194] Registered new tserver with Master: f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159)
I20260809 12:04:21.100968 11392 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:37043
WARNING: no policy specified for oryx/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab.
Entry for principal oryx/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab.
I20260809 12:04:21.164126 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40243
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab
--principal=oryx/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:42535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:21.512708 11879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:21.512995 11879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:21.513098 11879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:21.520424 11879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:21.520532 11879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:21.520627 11879 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:04:21.520709 11879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:04:21.530824 11879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:42535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab
--principal=oryx/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.147.126:40243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:21.532019 11879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:21.533248 11879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 12:04:21.537436 11677 heartbeater.cc:503] Master 127.0.147.126:40243 was elected leader, sending a full tablet report...
W20260809 12:04:21.546336 11885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:21.546300 11888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:21.546278 11886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:21.547564 11879 server_base.cc:1034] running on GCE node
I20260809 12:04:21.548276 11879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:21.549185 11879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:21.550462 11879 hybrid_clock.cc:648] HybridClock initialized: now 1786277061550376 us; error 57 us; skew 500 ppm
I20260809 12:04:21.550763 11879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:21 dist-test-slave-sb2z krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277061, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:21.553980 11879 init.cc:377] Logged in from keytab as oryx/127.0.147.67@KRBTEST.COM (short username oryx)
I20260809 12:04:21.555497 11879 webserver.cc:460] Webserver started at http://127.0.147.67:35779/ using document root <none> and password file <none>
I20260809 12:04:21.556250 11879 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:21.556384 11879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:21.556711 11879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:21.559670 11879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "0cae805193bc45509e7e7328163ce4f9"
format_stamp: "Formatted at 2026-08-09 12:04:21 on dist-test-slave-sb2z"
server_key: "72690bd2dcb120e897586d8bdf074dcf"
server_key_iv: "67164a5c2cd08172a3249b70bb3f1f03"
server_key_version: "encryptionkey@0"
I20260809 12:04:21.560806 11879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "0cae805193bc45509e7e7328163ce4f9"
format_stamp: "Formatted at 2026-08-09 12:04:21 on dist-test-slave-sb2z"
server_key: "72690bd2dcb120e897586d8bdf074dcf"
server_key_iv: "67164a5c2cd08172a3249b70bb3f1f03"
server_key_version: "encryptionkey@0"
I20260809 12:04:21.566109 11879 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:04:21.569582 11898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:21.570883 11879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:04:21.571071 11879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "0cae805193bc45509e7e7328163ce4f9"
format_stamp: "Formatted at 2026-08-09 12:04:21 on dist-test-slave-sb2z"
server_key: "72690bd2dcb120e897586d8bdf074dcf"
server_key_iv: "67164a5c2cd08172a3249b70bb3f1f03"
server_key_version: "encryptionkey@0"
I20260809 12:04:21.571303 11879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:21.588238 11879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:21.592206 11879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:21.592473 11879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:21.595089 11879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:21.595217 11879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:21.595393 11879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:21.595472 11879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:04:21 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277061, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:21.619714 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:21.595442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:42839 (local address 127.0.147.126:40243)
0809 12:04:21.596018 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:21.596025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:21.597261 (+ 1236us) server_negotiation.cc:407] Connection header received
0809 12:04:21.599503 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:21.599510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:21.599664 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:21.599827 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:21.601909 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.602876 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.605348 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.605966 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.606107 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:21.611218 (+ 5111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:21.611224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:21.611227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:21.611263 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:21.615494 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:21.616695 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:21.616704 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:21.616710 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:21.616897 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:21.617621 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:21.617627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:21.617631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:21.617928 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:21.618053 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:21.619134 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0809 12:04:21.619316 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":256,"threads_started":1}
I20260809 12:04:21.621591 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:21.596175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40243 (local address 127.0.147.67:42839)
0809 12:04:21.597071 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:21.597128 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:21.599151 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:21.600124 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:21.600140 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:21.600873 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:21.601710 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:21.601747 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.603064 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.603068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:21.605202 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:21.605221 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.606233 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.606241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:21.606450 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:21.607317 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:21.607348 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:21.610911 (+ 3563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:21.615749 (+ 4838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:21.615763 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:21.615786 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:21.616443 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:21.617064 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:21.617073 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:21.617078 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:21.617421 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:21.618066 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:21.618077 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:21.618639 (+ 562us) client_negotiation.cc:769] Sending connection context
0809 12:04:21.619011 (+ 372us) client_negotiation.cc:240] Negotiation successful
0809 12:04:21.619735 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":436,"threads_started":1}
I20260809 12:04:21.646696 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:21.629624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:59399 (local address 127.0.147.126:40243)
0809 12:04:21.629818 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:21.629824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:21.630223 (+ 399us) server_negotiation.cc:407] Connection header received
0809 12:04:21.630552 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:21.630567 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:21.630661 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:21.630777 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:21.632514 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.633848 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.636774 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.637637 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.637755 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:21.640803 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:21.640812 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:21.640817 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:21.640857 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:21.644008 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:21.644669 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:21.644679 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:21.644681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:21.644801 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:21.645185 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:21.645190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:21.645200 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:21.645470 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:21.645646 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:21.646026 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 12:04:21.646315 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:04:21.646770 11999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:21.629243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40243 (local address 127.0.147.67:59399)
0809 12:04:21.630093 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:21.630119 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:21.630392 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:21.630837 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:21.630848 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:21.631747 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:21.632278 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:21.632299 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.634089 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.634100 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:21.636534 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:21.636555 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.638113 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.638125 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:21.638324 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:21.639018 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:21.639049 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:21.640573 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:21.644144 (+ 3571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:21.644151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:21.644155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:21.644537 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:21.644916 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:21.644921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:21.644925 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:21.645066 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:21.645592 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:21.645597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:21.645782 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 12:04:21.645918 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 12:04:21.646258 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":483,"threads_started":1}
I20260809 12:04:21.654376 11879 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:41815
I20260809 12:04:21.654400 12063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:41815 every 8 connection(s)
I20260809 12:04:21.655697 11879 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:04:21.657661 12064 heartbeater.cc:347] Connected to a master server at 127.0.147.126:40243
I20260809 12:04:21.657887 12064 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:21.658706 12064 heartbeater.cc:511] Master 127.0.147.126:40243 requested a full tablet report, sending...
I20260809 12:04:21.660212 11392 ts_manager.cc:194] Registered new tserver with Master: 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67:41815)
I20260809 12:04:21.661165 11392 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:42839
I20260809 12:04:21.663130 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 11879
I20260809 12:04:21.663300 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:04:21.663833 589 external_mini_cluster.cc:1428] Setting key 584321f8f69b0ac2bd7247a1f52d67e5
I20260809 12:04:21.666863 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:04:21 dist-test-slave-sb2z krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277061, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:04:21 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:21 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:22.103367 11875 heartbeater.cc:503] Master 127.0.147.126:40243 was elected leader, sending a full tablet report...
I20260809 12:04:22.663740 12064 heartbeater.cc:503] Master 127.0.147.126:40243 was elected leader, sending a full tablet report...
W20260809 12:04:22.898239 12074 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.173s user 0.000s sys 0.021s
W20260809 12:04:22.899574 11909 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:21.719622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.0.147.126:40243)
0809 12:04:21.719808 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:21.719813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:21.720063 (+ 250us) server_negotiation.cc:407] Connection header received
0809 12:04:21.720241 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:21.720246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:21.720314 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:21.720414 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:21.721718 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.722467 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.724006 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:21.724457 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:21.724607 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:22.899191 (+1174584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53620: BlockingRecv error: recv got EOF from 127.0.0.1:53620 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 12:04:22 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277061, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:22.918012 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:22.904000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.0.147.126:40243)
0809 12:04:22.904176 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:22.904192 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:22.904590 (+ 398us) server_negotiation.cc:407] Connection header received
0809 12:04:22.904863 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:22.904868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:22.904941 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:22.905063 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:22.906355 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.907085 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.908577 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.909026 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.909177 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:22.912966 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:22.912973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:22.912977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:22.913021 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:22.915434 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:22.916100 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:22.916106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:22.916109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:22.916236 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:22.916621 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:22.916626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:22.916630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:22.916904 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:22.917178 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:22.917489 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 12:04:22.917706 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 12:04:22.922092 11392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:04:22.929189 11392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:04:22.951063 12088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:22.940956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:38117 (local address 127.0.0.1:48568)
0809 12:04:22.941950 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:22.941995 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:22.942200 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:22.944155 (+ 1955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:22.944162 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:22.944193 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:22.944601 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:22.944617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.946557 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.946562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:22.948746 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:22.948758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.950394 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.950399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:22.950539 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:22.950574 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:04:22.950659 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 12:04:22.950775 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":64,"thread_start_us":1535,"threads_started":1}
I20260809 12:04:22.952270 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:22.941151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48568 (local address 127.0.147.65:38117)
0809 12:04:22.943476 (+ 2325us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:22.943485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:22.943544 (+ 59us) server_negotiation.cc:407] Connection header received
0809 12:04:22.943648 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:22.943655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:22.943918 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:22.944836 (+ 918us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:22.945052 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.946396 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.948902 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.950164 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.951566 (+ 1402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:22.951755 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:22.951807 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 12:04:22.951984 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2150,"thread_start_us":2004,"threads_started":1}
I20260809 12:04:22.952867 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:22.943518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45976 (local address 127.0.147.66:38159)
0809 12:04:22.944125 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:22.944134 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:22.944160 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:04:22.944287 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:22.944297 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:22.944513 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:22.944740 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:22.945375 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.947035 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.950129 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.951743 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.951912 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:22.952126 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:22.952353 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 12:04:22.952547 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":300,"threads_started":1}
I20260809 12:04:22.953182 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:22.941056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:41815 (local address 127.0.0.1:33906)
0809 12:04:22.942779 (+ 1723us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:22.942801 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:22.942935 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:22.945116 (+ 2181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:22.945120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:22.945143 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:22.945614 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:22.945628 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.947481 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.947486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:22.950118 (+ 2632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:22.950129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.952152 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.952160 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:22.952350 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:22.952380 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 12:04:22.952449 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 12:04:22.952680 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1584,"mutex_wait_us":81,"thread_start_us":818,"threads_started":1}
I20260809 12:04:22.953631 12091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:22.942970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:38159 (local address 127.0.0.1:45976)
0809 12:04:22.943924 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:22.943943 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:22.944071 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:22.944731 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:22.944736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:22.944766 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:22.945139 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:22.945152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.947193 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.947199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:22.949944 (+ 2745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:22.949956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.951967 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.951973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:22.952151 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:22.952182 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:04:22.952291 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 12:04:22.952713 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":92,"thread_start_us":1075,"threads_started":1}
I20260809 12:04:22.954186 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:22.941423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.0.147.67:41815)
0809 12:04:22.944571 (+ 3148us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:22.944579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:22.944606 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:04:22.944704 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:22.944711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:22.944909 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:22.945117 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:22.945796 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.947329 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.950283 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:22.951843 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:22.953423 (+ 1580us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:22.953642 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:22.953709 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 12:04:22.953880 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2988,"thread_start_us":2855,"threads_started":1}
I20260809 12:04:22.956416 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 7fc0e2ac9da14d07a90ec3f82f19c919 (DEFAULT_TABLE table=test-table [id=ed6af7445a2245328dedac1d71d7a649]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:22.957724 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fc0e2ac9da14d07a90ec3f82f19c919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:22.957885 11971 tablet_service.cc:1467] Processing CreateTablet for tablet 7fc0e2ac9da14d07a90ec3f82f19c919 (DEFAULT_TABLE table=test-table [id=ed6af7445a2245328dedac1d71d7a649]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:22.958498 11781 tablet_service.cc:1467] Processing CreateTablet for tablet 7fc0e2ac9da14d07a90ec3f82f19c919 (DEFAULT_TABLE table=test-table [id=ed6af7445a2245328dedac1d71d7a649]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:22.959303 11971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fc0e2ac9da14d07a90ec3f82f19c919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:22.959858 11781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fc0e2ac9da14d07a90ec3f82f19c919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:22.969681 12095 tablet_bootstrap.cc:492] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: Bootstrap starting.
I20260809 12:04:22.973513 12095 tablet_bootstrap.cc:654] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:22.973780 12098 tablet_bootstrap.cc:492] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: Bootstrap starting.
I20260809 12:04:22.975035 12095 log.cc:824] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:22.977495 12098 tablet_bootstrap.cc:654] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:22.978673 12095 tablet_bootstrap.cc:492] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: No bootstrap required, opened a new log
I20260809 12:04:22.978778 12098 log.cc:824] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:22.978989 12095 ts_tablet_manager.cc:1397] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 12:04:22.976696 12099 tablet_bootstrap.cc:492] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: Bootstrap starting.
I20260809 12:04:22.981925 12098 tablet_bootstrap.cc:492] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: No bootstrap required, opened a new log
I20260809 12:04:22.982200 12098 ts_tablet_manager.cc:1397] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 12:04:22.982868 12099 tablet_bootstrap.cc:654] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:22.983876 12099 log.cc:824] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:22.986233 12099 tablet_bootstrap.cc:492] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: No bootstrap required, opened a new log
I20260809 12:04:22.986493 12099 ts_tablet_manager.cc:1397] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260809 12:04:22.990111 12095 raft_consensus.cc:348] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } }
I20260809 12:04:22.990535 12095 raft_consensus.cc:374] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:22.990638 12095 raft_consensus.cc:729] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd8d9fdd6fd4792b96d58d568301083, State: Initialized, Role: FOLLOWER
I20260809 12:04:22.991284 12095 consensus_queue.cc:260] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } }
I20260809 12:04:22.993726 12095 ts_tablet_manager.cc:1428] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260809 12:04:22.994055 12098 raft_consensus.cc:348] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } }
I20260809 12:04:22.994514 12098 raft_consensus.cc:374] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:22.994614 12098 raft_consensus.cc:729] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cae805193bc45509e7e7328163ce4f9, State: Initialized, Role: FOLLOWER
I20260809 12:04:22.994445 12099 raft_consensus.cc:348] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } }
I20260809 12:04:22.994837 12099 raft_consensus.cc:374] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:22.995117 12099 raft_consensus.cc:729] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d109cc3d55413c8fc4573f3db36192, State: Initialized, Role: FOLLOWER
I20260809 12:04:22.995198 12098 consensus_queue.cc:260] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } }
I20260809 12:04:22.995718 12099 consensus_queue.cc:260] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } }
I20260809 12:04:22.997126 12098 ts_tablet_manager.cc:1428] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260809 12:04:22.998682 12099 ts_tablet_manager.cc:1428] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
W20260809 12:04:23.030973 11679 tablet.cc:2369] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:04:23.037555 12103 raft_consensus.cc:482] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:04:23.037873 12103 raft_consensus.cc:504] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } }
I20260809 12:04:23.039561 12103 leader_election.cc:288] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67:41815), f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159)
I20260809 12:04:23.047254 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.040726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:40609 (local address 127.0.147.66:38159)
0809 12:04:23.040985 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.040994 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.041021 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:04:23.041114 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.041122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.041239 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.041365 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:23.041944 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.043040 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.045159 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.046266 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.046445 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.046518 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.046828 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.046978 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:04:23.047343 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.039881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:38159 (local address 127.0.147.65:40609)
0809 12:04:23.040755 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:23.040778 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:23.040921 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:23.041382 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:23.041385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:23.041405 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:23.041782 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.041792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.043177 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.043183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.044990 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.045003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.046465 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.046471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.046623 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.046646 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:04:23.046779 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 12:04:23.046903 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":479,"threads_started":1}
I20260809 12:04:23.048172 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.040091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:36223 (local address 127.0.147.67:41815)
0809 12:04:23.040629 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.040638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.042039 (+ 1401us) server_negotiation.cc:407] Connection header received
0809 12:04:23.042195 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.042199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.042272 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.042342 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:23.042946 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.044032 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.046024 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.046936 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.047103 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.047169 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.047790 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.047917 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370}
I20260809 12:04:23.048213 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.040114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:41815 (local address 127.0.147.65:36223)
0809 12:04:23.041917 (+ 1803us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:23.041934 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:23.042072 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:23.042397 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:23.042403 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:23.042428 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:23.042797 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.042807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.044179 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.044183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.045862 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.045877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.047189 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.047196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.047380 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.047424 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 12:04:23.047743 (+ 319us) client_negotiation.cc:240] Negotiation successful
0809 12:04:23.047869 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1684,"thread_start_us":1477,"threads_started":1}
I20260809 12:04:23.048439 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc0e2ac9da14d07a90ec3f82f19c919" candidate_uuid: "5bd8d9fdd6fd4792b96d58d568301083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192" is_pre_election: true
I20260809 12:04:23.049078 11814 raft_consensus.cc:2454] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bd8d9fdd6fd4792b96d58d568301083 in term 0.
I20260809 12:04:23.049454 12006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc0e2ac9da14d07a90ec3f82f19c919" candidate_uuid: "5bd8d9fdd6fd4792b96d58d568301083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cae805193bc45509e7e7328163ce4f9" is_pre_election: true
I20260809 12:04:23.049783 11510 leader_election.cc:302] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bd8d9fdd6fd4792b96d58d568301083, f9d109cc3d55413c8fc4573f3db36192; no voters:
I20260809 12:04:23.050078 12006 raft_consensus.cc:2454] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bd8d9fdd6fd4792b96d58d568301083 in term 0.
I20260809 12:04:23.050412 12103 raft_consensus.cc:2781] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:04:23.050565 12103 raft_consensus.cc:482] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:04:23.050674 12103 raft_consensus.cc:3037] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:23.053571 12103 raft_consensus.cc:504] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } }
I20260809 12:04:23.054391 12103 leader_election.cc:288] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 election: Requested vote from peers 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67:41815), f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159)
I20260809 12:04:23.054862 12006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc0e2ac9da14d07a90ec3f82f19c919" candidate_uuid: "5bd8d9fdd6fd4792b96d58d568301083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cae805193bc45509e7e7328163ce4f9"
I20260809 12:04:23.055090 12006 raft_consensus.cc:3037] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:23.055130 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc0e2ac9da14d07a90ec3f82f19c919" candidate_uuid: "5bd8d9fdd6fd4792b96d58d568301083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192"
I20260809 12:04:23.055338 11814 raft_consensus.cc:3037] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:23.058362 12006 raft_consensus.cc:2454] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bd8d9fdd6fd4792b96d58d568301083 in term 1.
I20260809 12:04:23.058460 11814 raft_consensus.cc:2454] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bd8d9fdd6fd4792b96d58d568301083 in term 1.
I20260809 12:04:23.058885 11510 leader_election.cc:302] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bd8d9fdd6fd4792b96d58d568301083, f9d109cc3d55413c8fc4573f3db36192; no voters:
I20260809 12:04:23.059160 12103 raft_consensus.cc:2781] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:23.059631 12103 raft_consensus.cc:686] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 LEADER]: Becoming Leader. State: Replica: 5bd8d9fdd6fd4792b96d58d568301083, State: Running, Role: LEADER
I20260809 12:04:23.060177 12103 consensus_queue.cc:237] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } }
I20260809 12:04:23.065132 11391 catalog_manager.cc:5495] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65). New cstate: current_term: 1 leader_uuid: "5bd8d9fdd6fd4792b96d58d568301083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } health_report { overall_health: UNKNOWN } } }
W20260809 12:04:23.098423 11876 tablet.cc:2369] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:04:23.107692 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.102031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.0.147.65:38117)
0809 12:04:23.102244 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.102251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.102367 (+ 116us) server_negotiation.cc:407] Connection header received
0809 12:04:23.102579 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.102584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.102728 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.102892 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:04:23.103256 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.104371 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.105735 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.106130 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.106225 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.106560 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:04:23.107107 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:04:23.107216 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.107290 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.107353 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:04:23.115645 11814 raft_consensus.cc:1261] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Refusing update from remote peer 5bd8d9fdd6fd4792b96d58d568301083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:23.115761 12006 raft_consensus.cc:1261] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Refusing update from remote peer 5bd8d9fdd6fd4792b96d58d568301083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:23.116681 12111 consensus_queue.cc:1035] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:23.117059 12103 consensus_queue.cc:1035] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:23.142038 11392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:23.142316 11392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:23.145696 11392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3: Sending DeleteTablet for 3 replicas of tablet 7fc0e2ac9da14d07a90ec3f82f19c919
I20260809 12:04:23.146904 11971 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fc0e2ac9da14d07a90ec3f82f19c919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:23 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:33906
I20260809 12:04:23.146944 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fc0e2ac9da14d07a90ec3f82f19c919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:23 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:48568
I20260809 12:04:23.147092 11781 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fc0e2ac9da14d07a90ec3f82f19c919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:23 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:45976
I20260809 12:04:23.149485 11392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:04:23.150310 11392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:04:23.161921 11578 tablet_service.cc:1467] Processing CreateTablet for tablet d71cb95bd30b4bc7b7b8e3b70816e9dd (DEFAULT_TABLE table=test-table [id=d34725fa438f422fbfcd2abc9b02c560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:23.162293 11970 tablet_service.cc:1467] Processing CreateTablet for tablet d71cb95bd30b4bc7b7b8e3b70816e9dd (DEFAULT_TABLE table=test-table [id=d34725fa438f422fbfcd2abc9b02c560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:23.162611 11578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71cb95bd30b4bc7b7b8e3b70816e9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:23.162876 11970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71cb95bd30b4bc7b7b8e3b70816e9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:23.170094 12095 tablet_bootstrap.cc:492] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: Bootstrap starting.
I20260809 12:04:23.170498 12098 tablet_bootstrap.cc:492] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: Bootstrap starting.
I20260809 12:04:23.172441 12095 tablet_bootstrap.cc:654] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:23.172565 12098 tablet_bootstrap.cc:654] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:23.178574 12129 tablet_replica.cc:330] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: stopping tablet replica
I20260809 12:04:23.179159 12129 raft_consensus.cc:2229] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:23.179857 12129 raft_consensus.cc:2258] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:23.182257 12098 tablet_bootstrap.cc:492] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: No bootstrap required, opened a new log
I20260809 12:04:23.182412 12098 ts_tablet_manager.cc:1397] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260809 12:04:23.183068 12129 ts_tablet_manager.cc:1905] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:23.183279 12098 raft_consensus.cc:348] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.183511 12098 raft_consensus.cc:374] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:23.183648 12098 raft_consensus.cc:729] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cae805193bc45509e7e7328163ce4f9, State: Initialized, Role: FOLLOWER
I20260809 12:04:23.183983 12098 consensus_queue.cc:260] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.185186 12098 ts_tablet_manager.cc:1428] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 12:04:23.185433 11779 tablet_service.cc:1467] Processing CreateTablet for tablet d71cb95bd30b4bc7b7b8e3b70816e9dd (DEFAULT_TABLE table=test-table [id=d34725fa438f422fbfcd2abc9b02c560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:23.186030 11779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71cb95bd30b4bc7b7b8e3b70816e9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:23.189724 12127 tablet_replica.cc:330] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: stopping tablet replica
I20260809 12:04:23.190213 12127 raft_consensus.cc:2229] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:04:23.190754 12095 tablet_bootstrap.cc:492] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: No bootstrap required, opened a new log
I20260809 12:04:23.190934 12095 ts_tablet_manager.cc:1397] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.004s
I20260809 12:04:23.190889 12127 raft_consensus.cc:2258] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:23.191658 12095 raft_consensus.cc:348] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.191881 12095 raft_consensus.cc:374] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:23.191994 12095 raft_consensus.cc:729] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd8d9fdd6fd4792b96d58d568301083, State: Initialized, Role: FOLLOWER
I20260809 12:04:23.192324 12095 consensus_queue.cc:260] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.192651 12131 tablet_replica.cc:330] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: stopping tablet replica
I20260809 12:04:23.192792 12099 tablet_bootstrap.cc:492] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: Bootstrap starting.
I20260809 12:04:23.193099 12131 raft_consensus.cc:2229] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:23.193624 12131 raft_consensus.cc:2258] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:23.194552 12099 tablet_bootstrap.cc:654] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:23.196280 12095 ts_tablet_manager.cc:1428] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 12:04:23.197382 12127 ts_tablet_manager.cc:1905] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:23.198340 12129 ts_tablet_manager.cc:1918] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:23.198501 12129 log.cc:1196] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/7fc0e2ac9da14d07a90ec3f82f19c919
I20260809 12:04:23.199067 12129 ts_tablet_manager.cc:1939] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 0cae805193bc45509e7e7328163ce4f9: Deleting consensus metadata
I20260809 12:04:23.199751 12131 ts_tablet_manager.cc:1905] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:23.201128 11374 catalog_manager.cc:4841] TS 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67:41815): tablet 7fc0e2ac9da14d07a90ec3f82f19c919 (table test-table [id=ed6af7445a2245328dedac1d71d7a649]) successfully deleted
I20260809 12:04:23.201453 12111 raft_consensus.cc:482] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:04:23.201651 12111 raft_consensus.cc:504] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.202417 12111 leader_election.cc:288] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159), 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67:41815)
I20260809 12:04:23.203173 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71cb95bd30b4bc7b7b8e3b70816e9dd" candidate_uuid: "5bd8d9fdd6fd4792b96d58d568301083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192" is_pre_election: true
I20260809 12:04:23.204330 12005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71cb95bd30b4bc7b7b8e3b70816e9dd" candidate_uuid: "5bd8d9fdd6fd4792b96d58d568301083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cae805193bc45509e7e7328163ce4f9" is_pre_election: true
I20260809 12:04:23.204643 12005 raft_consensus.cc:2454] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bd8d9fdd6fd4792b96d58d568301083 in term 0.
I20260809 12:04:23.205226 11517 leader_election.cc:302] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cae805193bc45509e7e7328163ce4f9, 5bd8d9fdd6fd4792b96d58d568301083; no voters:
W20260809 12:04:23.205879 11510 leader_election.cc:341] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159): Illegal state: must be running to vote when last-logged opid is not known
I20260809 12:04:23.206357 12111 raft_consensus.cc:2781] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:04:23.206471 12111 raft_consensus.cc:482] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:04:23.206650 12111 raft_consensus.cc:3037] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:23.208247 12127 ts_tablet_manager.cc:1918] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:23.209138 12127 log.cc:1196] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/7fc0e2ac9da14d07a90ec3f82f19c919
I20260809 12:04:23.209815 12099 tablet_bootstrap.cc:492] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: No bootstrap required, opened a new log
I20260809 12:04:23.209841 12127 ts_tablet_manager.cc:1939] T 7fc0e2ac9da14d07a90ec3f82f19c919 P 5bd8d9fdd6fd4792b96d58d568301083: Deleting consensus metadata
I20260809 12:04:23.209982 12099 ts_tablet_manager.cc:1397] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.005s
I20260809 12:04:23.209869 12111 raft_consensus.cc:504] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.210925 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71cb95bd30b4bc7b7b8e3b70816e9dd" candidate_uuid: "5bd8d9fdd6fd4792b96d58d568301083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192"
W20260809 12:04:23.211508 11510 leader_election.cc:341] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159): Illegal state: must be running to vote when last-logged opid is not known
I20260809 12:04:23.211908 11368 catalog_manager.cc:4841] TS 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117): tablet 7fc0e2ac9da14d07a90ec3f82f19c919 (table test-table [id=ed6af7445a2245328dedac1d71d7a649]) successfully deleted
I20260809 12:04:23.211863 12099 raft_consensus.cc:348] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.212117 12099 raft_consensus.cc:374] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:23.212256 12099 raft_consensus.cc:729] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d109cc3d55413c8fc4573f3db36192, State: Initialized, Role: FOLLOWER
I20260809 12:04:23.212535 12099 consensus_queue.cc:260] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.212334 12111 leader_election.cc:288] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 election: Requested vote from peers f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159), 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67:41815)
I20260809 12:04:23.212900 12005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71cb95bd30b4bc7b7b8e3b70816e9dd" candidate_uuid: "5bd8d9fdd6fd4792b96d58d568301083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cae805193bc45509e7e7328163ce4f9"
I20260809 12:04:23.213111 12005 raft_consensus.cc:3037] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:23.213310 12099 ts_tablet_manager.cc:1428] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 12:04:23.215967 12005 raft_consensus.cc:2454] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bd8d9fdd6fd4792b96d58d568301083 in term 1.
I20260809 12:04:23.216415 11517 leader_election.cc:302] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cae805193bc45509e7e7328163ce4f9, 5bd8d9fdd6fd4792b96d58d568301083; no voters: f9d109cc3d55413c8fc4573f3db36192
I20260809 12:04:23.216861 12111 raft_consensus.cc:2781] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:23.217140 12111 raft_consensus.cc:686] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 LEADER]: Becoming Leader. State: Replica: 5bd8d9fdd6fd4792b96d58d568301083, State: Running, Role: LEADER
I20260809 12:04:23.217401 12111 consensus_queue.cc:237] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.220038 12131 ts_tablet_manager.cc:1918] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:23.220207 12131 log.cc:1196] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/7fc0e2ac9da14d07a90ec3f82f19c919
I20260809 12:04:23.220126 11391 catalog_manager.cc:5495] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65). New cstate: current_term: 1 leader_uuid: "5bd8d9fdd6fd4792b96d58d568301083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } health_report { overall_health: UNKNOWN } } }
I20260809 12:04:23.220811 12131 ts_tablet_manager.cc:1939] T 7fc0e2ac9da14d07a90ec3f82f19c919 P f9d109cc3d55413c8fc4573f3db36192: Deleting consensus metadata
I20260809 12:04:23.223978 11368 catalog_manager.cc:4841] TS f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159): tablet 7fc0e2ac9da14d07a90ec3f82f19c919 (table test-table [id=ed6af7445a2245328dedac1d71d7a649]) successfully deleted
Aug 09 12:04:23 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277059, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.126@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:23.274112 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.258237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40243 (local address 127.0.0.1:53636)
0809 12:04:23.258952 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:23.258975 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:23.259162 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:23.259513 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:23.259518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:23.260513 (+ 995us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:23.260868 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.260883 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.261943 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.261949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.263569 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.263584 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.264330 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.264338 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.264511 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.265210 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:23.265232 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:23.267889 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:23.271708 (+ 3819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:23.271718 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:23.271724 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:23.272149 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:23.272539 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:23.272542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:23.272544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:23.272691 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:23.273306 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:23.273312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:23.273443 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 12:04:23.273574 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 12:04:23.273763 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":385,"threads_started":1}
I20260809 12:04:23.274410 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.258215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.0.147.126:40243)
0809 12:04:23.258512 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.258521 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.259092 (+ 571us) server_negotiation.cc:407] Connection header received
0809 12:04:23.259288 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.259292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.259353 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.259440 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:23.261027 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.261813 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.263710 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.264131 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.264231 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.268088 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:23.268095 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:23.268100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:23.268137 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:23.271579 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:23.272282 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:23.272288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:23.272292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:23.272434 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:23.272809 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:23.272814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:23.272818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:23.273184 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:23.273332 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.273724 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.273954 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260809 12:04:23.280157 11391 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:53636:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 12:04:23.281243 11391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:04:23.288269 11971 tablet_service.cc:1467] Processing CreateTablet for tablet c74f179f2bbd4dbcb63d7fd4545fd37c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0be091b5dd364d3ea58c9bee3e0f3cb0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:04:23.288349 11779 tablet_service.cc:1467] Processing CreateTablet for tablet c74f179f2bbd4dbcb63d7fd4545fd37c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0be091b5dd364d3ea58c9bee3e0f3cb0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:04:23.288362 11580 tablet_service.cc:1467] Processing CreateTablet for tablet c74f179f2bbd4dbcb63d7fd4545fd37c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0be091b5dd364d3ea58c9bee3e0f3cb0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:04:23.288869 11779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c74f179f2bbd4dbcb63d7fd4545fd37c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:23.288892 11971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c74f179f2bbd4dbcb63d7fd4545fd37c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:23.288937 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c74f179f2bbd4dbcb63d7fd4545fd37c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:23.296327 12099 tablet_bootstrap.cc:492] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: Bootstrap starting.
I20260809 12:04:23.296473 12095 tablet_bootstrap.cc:492] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: Bootstrap starting.
I20260809 12:04:23.297849 12099 tablet_bootstrap.cc:654] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:23.298175 12098 tablet_bootstrap.cc:492] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: Bootstrap starting.
I20260809 12:04:23.298341 12095 tablet_bootstrap.cc:654] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:23.300374 12098 tablet_bootstrap.cc:654] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:23.306389 12098 tablet_bootstrap.cc:492] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: No bootstrap required, opened a new log
I20260809 12:04:23.306582 12098 ts_tablet_manager.cc:1397] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260809 12:04:23.307077 12095 tablet_bootstrap.cc:492] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: No bootstrap required, opened a new log
I20260809 12:04:23.307246 12095 ts_tablet_manager.cc:1397] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.005s
I20260809 12:04:23.307264 12098 raft_consensus.cc:348] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.307524 12098 raft_consensus.cc:374] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:23.307657 12098 raft_consensus.cc:729] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cae805193bc45509e7e7328163ce4f9, State: Initialized, Role: FOLLOWER
I20260809 12:04:23.307929 12098 consensus_queue.cc:260] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.308188 12095 raft_consensus.cc:348] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.308444 12095 raft_consensus.cc:374] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:23.308528 12095 raft_consensus.cc:729] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd8d9fdd6fd4792b96d58d568301083, State: Initialized, Role: FOLLOWER
I20260809 12:04:23.308534 12098 ts_tablet_manager.cc:1428] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 12:04:23.308801 12095 consensus_queue.cc:260] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.308858 12104 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.309270 12104 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:04:23.309242 12111 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.309528 12111 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:04:23.310616 12095 ts_tablet_manager.cc:1428] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 12:04:23.311429 12099 tablet_bootstrap.cc:492] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: No bootstrap required, opened a new log
I20260809 12:04:23.311640 12099 ts_tablet_manager.cc:1397] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.008s
I20260809 12:04:23.312443 12099 raft_consensus.cc:348] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.312693 12099 raft_consensus.cc:374] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:23.312788 12099 raft_consensus.cc:729] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d109cc3d55413c8fc4573f3db36192, State: Initialized, Role: FOLLOWER
I20260809 12:04:23.313081 12099 consensus_queue.cc:260] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.313551 12106 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.313947 12106 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:04:23.314577 12099 ts_tablet_manager.cc:1428] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 12:04:23.372196 12104 raft_consensus.cc:482] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:04:23.372417 12104 raft_consensus.cc:504] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.374042 12104 leader_election.cc:288] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117), f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159)
I20260809 12:04:23.381983 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.374482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:55405 (local address 127.0.147.65:38117)
0809 12:04:23.374672 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.374679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.375161 (+ 482us) server_negotiation.cc:407] Connection header received
0809 12:04:23.375464 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.375469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.375586 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.375663 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:23.376750 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.377967 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.379957 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.380897 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.381095 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.381176 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.381497 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.381644 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:04:23.382006 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.374451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:38117 (local address 127.0.147.67:55405)
0809 12:04:23.375042 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:23.375059 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:23.375263 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:23.376119 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:23.376123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:23.376155 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:23.376551 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.376562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.378123 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.378128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.379784 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.379796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.381138 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.381144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.381314 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.381349 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:04:23.381450 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 12:04:23.381578 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":1,"spinlock_wait_cycles":12160,"thread_start_us":328,"threads_started":1}
I20260809 12:04:23.383116 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.375225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:38159 (local address 127.0.147.67:49073)
0809 12:04:23.376119 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:23.376133 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:23.376318 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:23.376665 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:23.376669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:23.376694 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:23.377113 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.377143 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.378301 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.378308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.380491 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.380506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.382304 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.382310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.382483 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.382521 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:04:23.382617 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 12:04:23.382739 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"spinlock_wait_cycles":1920,"thread_start_us":580,"threads_started":1}
I20260809 12:04:23.383121 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.375601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:49073 (local address 127.0.147.66:38159)
0809 12:04:23.376015 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.376022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.376221 (+ 199us) server_negotiation.cc:407] Connection header received
0809 12:04:23.376473 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.376478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.376549 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.376647 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:23.377302 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.378151 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.380659 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.381885 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.382067 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.382143 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.382645 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.382819 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268}
I20260809 12:04:23.383484 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c74f179f2bbd4dbcb63d7fd4545fd37c" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd8d9fdd6fd4792b96d58d568301083" is_pre_election: true
I20260809 12:04:23.384023 11609 raft_consensus.cc:2454] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 0.
I20260809 12:04:23.384802 11899 leader_election.cc:302] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cae805193bc45509e7e7328163ce4f9, 5bd8d9fdd6fd4792b96d58d568301083; no voters:
I20260809 12:04:23.385232 12104 raft_consensus.cc:2781] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:04:23.385385 12104 raft_consensus.cc:482] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:04:23.385491 12104 raft_consensus.cc:3037] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:23.385684 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c74f179f2bbd4dbcb63d7fd4545fd37c" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192" is_pre_election: true
I20260809 12:04:23.385941 11814 raft_consensus.cc:2454] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 0.
I20260809 12:04:23.388118 12104 raft_consensus.cc:504] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.388669 12104 leader_election.cc:288] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 election: Requested vote from peers 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117), f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159)
I20260809 12:04:23.389187 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c74f179f2bbd4dbcb63d7fd4545fd37c" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192"
I20260809 12:04:23.389231 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c74f179f2bbd4dbcb63d7fd4545fd37c" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd8d9fdd6fd4792b96d58d568301083"
I20260809 12:04:23.389384 11814 raft_consensus.cc:3037] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:23.389444 11609 raft_consensus.cc:3037] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:23.391386 11814 raft_consensus.cc:2454] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 1.
I20260809 12:04:23.391819 11899 leader_election.cc:302] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cae805193bc45509e7e7328163ce4f9, f9d109cc3d55413c8fc4573f3db36192; no voters:
I20260809 12:04:23.392089 12104 raft_consensus.cc:2781] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:23.392606 12104 raft_consensus.cc:686] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [term 1 LEADER]: Becoming Leader. State: Replica: 0cae805193bc45509e7e7328163ce4f9, State: Running, Role: LEADER
I20260809 12:04:23.392931 11609 raft_consensus.cc:2454] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 1.
I20260809 12:04:23.393009 12104 consensus_queue.cc:237] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.395440 12155 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: TxnStatusTablet state changed. Reason: New leader 0cae805193bc45509e7e7328163ce4f9. Latest consensus state: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.395884 12155 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:04:23.396755 12157 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 12:04:23.396981 12157 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 12:04:23.397361 11391 catalog_manager.cc:5495] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67). New cstate: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } health_report { overall_health: HEALTHY } } }
W20260809 12:04:23.409935 12065 tablet.cc:2369] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:04:23.417645 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.411925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33910 (local address 127.0.147.67:41815)
0809 12:04:23.412117 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.412124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.412403 (+ 279us) server_negotiation.cc:407] Connection header received
0809 12:04:23.412692 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.412698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.412784 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.412947 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:04:23.413331 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.414121 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.415379 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.415875 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.416032 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.416516 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:04:23.416967 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:04:23.417139 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.417223 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.417311 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:04:23.417668 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.411977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:41815 (local address 127.0.0.1:33910)
0809 12:04:23.412306 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:23.412322 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:23.412482 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:23.412968 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:23.412973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:23.412985 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:04:23.413195 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.413204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.414254 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.414261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.415235 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.415246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.416039 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.416045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.416254 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.416312 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:04:23.417097 (+ 785us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:04:23.417119 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 12:04:23.417205 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 12:04:23.417350 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200}
I20260809 12:04:23.421554 11814 raft_consensus.cc:1261] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Refusing update from remote peer 0cae805193bc45509e7e7328163ce4f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:23.422118 11609 raft_consensus.cc:1261] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Refusing update from remote peer 0cae805193bc45509e7e7328163ce4f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:23.422926 12104 consensus_queue.cc:1035] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:23.423725 12155 consensus_queue.cc:1035] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:23.428608 12106 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: TxnStatusTablet state changed. Reason: New leader 0cae805193bc45509e7e7328163ce4f9. Latest consensus state: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.428975 12106 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:04:23.431044 12155 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.431286 12155 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:04:23.433691 12106 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.433920 12106 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P f9d109cc3d55413c8fc4573f3db36192: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:04:23.434389 12116 mvcc.cc:205] Tried to move back new op lower bound from 7316590851767521280 to 7316590851662426112. Current Snapshot: MvccSnapshot[applied={T|T < 7316590851767521280 or (T in {7316590851767521280})}]
I20260809 12:04:23.456946 12111 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: TxnStatusTablet state changed. Reason: New leader 0cae805193bc45509e7e7328163ce4f9. Latest consensus state: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.457309 12111 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:04:23.460000 12104 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.460323 12104 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P 0cae805193bc45509e7e7328163ce4f9: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:04:23.465173 12111 tablet_replica.cc:439] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } } }
I20260809 12:04:23.465431 12111 tablet_replica.cc:442] T c74f179f2bbd4dbcb63d7fd4545fd37c P 5bd8d9fdd6fd4792b96d58d568301083: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:04:23.471047 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.461564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:32865 (local address 127.0.147.67:41815)
0809 12:04:23.461762 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.461771 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.462543 (+ 772us) server_negotiation.cc:407] Connection header received
0809 12:04:23.462874 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.462880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.462981 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.463065 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:04:23.464467 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.465632 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.468318 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.469020 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.469136 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.469854 (+ 718us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:04:23.469986 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:04:23.470074 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.470630 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.470710 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:04:23.471390 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.461624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:41815 (local address 127.0.147.65:32865)
0809 12:04:23.462439 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:23.462458 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:23.462679 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:23.463738 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:23.463742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:23.463758 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:04:23.464207 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.464222 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.466253 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.466259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.468127 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.468142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.469321 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.469328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.469553 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.469636 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:04:23.470478 (+ 842us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:04:23.470488 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:04:23.470579 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 12:04:23.470713 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":8504,"threads_started":1}
I20260809 12:04:23.482185 12005 raft_consensus.cc:1261] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Refusing update from remote peer 5bd8d9fdd6fd4792b96d58d568301083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:23.482810 12111 consensus_queue.cc:1035] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:23.483471 11814 raft_consensus.cc:3037] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:23.487399 11814 raft_consensus.cc:1261] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Refusing update from remote peer 5bd8d9fdd6fd4792b96d58d568301083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:23.488384 12111 consensus_queue.cc:1035] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:23.501478 12114 mvcc.cc:205] Tried to move back new op lower bound from 7316590852016861184 to 7316590850940960768. Current Snapshot: MvccSnapshot[applied={T|T < 7316590852016861184}]
I20260809 12:04:23.504597 12118 mvcc.cc:205] Tried to move back new op lower bound from 7316590852016861184 to 7316590850940960768. Current Snapshot: MvccSnapshot[applied={T|T < 7316590852016861184}]
I20260809 12:04:23.566948 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.550013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:36097 (local address 127.0.147.65:38117)
0809 12:04:23.550204 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.550213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.560227 (+ 10014us) server_negotiation.cc:407] Connection header received
0809 12:04:23.560477 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.560481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.560565 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.560683 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:04:23.561257 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.562129 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.565079 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.565545 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.565684 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.566144 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:04:23.566242 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:04:23.566363 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.566535 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.566613 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:04:23.567093 12173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.549498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:38117 (local address 127.0.147.67:36097)
0809 12:04:23.560058 (+ 10560us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:23.560093 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:23.560316 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:23.560699 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:23.560704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:23.560716 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:04:23.561090 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.561100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.562270 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.562281 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.564930 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:23.564943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.565744 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.565752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:23.565890 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.565945 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:04:23.566369 (+ 424us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:04:23.566383 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 12:04:23.566489 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:04:23.566623 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10300,"thread_start_us":10150,"threads_started":1}
I20260809 12:04:23.620949 11391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:23.621225 11391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:23.625205 11391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3: Sending DeleteTablet for 3 replicas of tablet d71cb95bd30b4bc7b7b8e3b70816e9dd
I20260809 12:04:23.626315 11779 tablet_service.cc:1514] Processing DeleteTablet for tablet d71cb95bd30b4bc7b7b8e3b70816e9dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:23 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:45976
I20260809 12:04:23.626477 11971 tablet_service.cc:1514] Processing DeleteTablet for tablet d71cb95bd30b4bc7b7b8e3b70816e9dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:23 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:33906
I20260809 12:04:23.626636 12131 tablet_replica.cc:330] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: stopping tablet replica
I20260809 12:04:23.626801 12129 tablet_replica.cc:330] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: stopping tablet replica
I20260809 12:04:23.626848 12131 raft_consensus.cc:2229] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:23.627029 12129 raft_consensus.cc:2229] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:23.627071 12131 raft_consensus.cc:2258] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:23.627236 12129 raft_consensus.cc:2258] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:23.629161 12129 ts_tablet_manager.cc:1905] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:23.634331 12131 ts_tablet_manager.cc:1905] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:23.642244 12131 ts_tablet_manager.cc:1918] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:04:23.642424 12131 log.cc:1196] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/d71cb95bd30b4bc7b7b8e3b70816e9dd
I20260809 12:04:23.642882 12131 ts_tablet_manager.cc:1939] T d71cb95bd30b4bc7b7b8e3b70816e9dd P f9d109cc3d55413c8fc4573f3db36192: Deleting consensus metadata
I20260809 12:04:23.644059 11368 catalog_manager.cc:4841] TS f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159): tablet d71cb95bd30b4bc7b7b8e3b70816e9dd (table test-table [id=d34725fa438f422fbfcd2abc9b02c560]) successfully deleted
I20260809 12:04:23.646688 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet d71cb95bd30b4bc7b7b8e3b70816e9dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:23 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:48568
I20260809 12:04:23.647040 12127 tablet_replica.cc:330] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: stopping tablet replica
I20260809 12:04:23.647253 12127 raft_consensus.cc:2229] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:04:23.647673 12127 raft_consensus.cc:2258] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:23.649153 12127 ts_tablet_manager.cc:1905] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:23.654570 12129 ts_tablet_manager.cc:1918] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:04:23.654747 12129 log.cc:1196] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/d71cb95bd30b4bc7b7b8e3b70816e9dd
I20260809 12:04:23.655263 12129 ts_tablet_manager.cc:1939] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 0cae805193bc45509e7e7328163ce4f9: Deleting consensus metadata
I20260809 12:04:23.656558 11374 catalog_manager.cc:4841] TS 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67:41815): tablet d71cb95bd30b4bc7b7b8e3b70816e9dd (table test-table [id=d34725fa438f422fbfcd2abc9b02c560]) successfully deleted
I20260809 12:04:23.662559 12127 ts_tablet_manager.cc:1918] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:04:23.662731 12127 log.cc:1196] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/d71cb95bd30b4bc7b7b8e3b70816e9dd
I20260809 12:04:23.663159 12127 ts_tablet_manager.cc:1939] T d71cb95bd30b4bc7b7b8e3b70816e9dd P 5bd8d9fdd6fd4792b96d58d568301083: Deleting consensus metadata
I20260809 12:04:23.664332 11368 catalog_manager.cc:4841] TS 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117): tablet d71cb95bd30b4bc7b7b8e3b70816e9dd (table test-table [id=d34725fa438f422fbfcd2abc9b02c560]) successfully deleted
Aug 09 12:04:23 dist-test-slave-sb2z krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:04:23 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:23 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:04:23.719906 11909 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:23.708628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.0.147.126:40243)
0809 12:04:23.708912 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.708921 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.709478 (+ 557us) server_negotiation.cc:407] Connection header received
0809 12:04:23.709728 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.709737 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.709817 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.709933 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:23.711372 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.712226 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.713482 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.713894 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.714037 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.719715 (+ 5678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53640: BlockingRecv error: recv got EOF from 127.0.0.1:53640 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 09 12:04:23 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM
I20260809 12:04:23.738646 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:23.723775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53656 (local address 127.0.147.126:40243)
0809 12:04:23.723991 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:23.723997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:23.724803 (+ 806us) server_negotiation.cc:407] Connection header received
0809 12:04:23.725087 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:23.725093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:23.725193 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:23.725290 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:23.726453 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.727148 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.728520 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:23.729004 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:23.729152 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:23.732921 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:23.732926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:23.732929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:23.732965 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:23.735958 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:23.736585 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:23.736591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:23.736595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:23.736733 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:23.737146 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:23.737158 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:23.737160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:23.737424 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:23.737699 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:23.738007 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 12:04:23.738250 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 12:04:23.742415 11391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:04:23.743070 11391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:04:23.750182 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 853a8907af9a412daa1e9a7a73f2fe9b (DEFAULT_TABLE table=test-table [id=53d36dc542ab4acd82f97fa4a13b6424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:23.750470 11779 tablet_service.cc:1467] Processing CreateTablet for tablet 853a8907af9a412daa1e9a7a73f2fe9b (DEFAULT_TABLE table=test-table [id=53d36dc542ab4acd82f97fa4a13b6424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:23.750684 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 853a8907af9a412daa1e9a7a73f2fe9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:23.750941 11779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 853a8907af9a412daa1e9a7a73f2fe9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:23.756533 11971 tablet_service.cc:1467] Processing CreateTablet for tablet 853a8907af9a412daa1e9a7a73f2fe9b (DEFAULT_TABLE table=test-table [id=53d36dc542ab4acd82f97fa4a13b6424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:23.756989 11971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 853a8907af9a412daa1e9a7a73f2fe9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:23.757354 12099 tablet_bootstrap.cc:492] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: Bootstrap starting.
I20260809 12:04:23.759027 12099 tablet_bootstrap.cc:654] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:23.773872 12095 tablet_bootstrap.cc:492] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: Bootstrap starting.
I20260809 12:04:23.775388 12099 tablet_bootstrap.cc:492] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: No bootstrap required, opened a new log
I20260809 12:04:23.775630 12099 ts_tablet_manager.cc:1397] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.016s
I20260809 12:04:23.776243 12095 tablet_bootstrap.cc:654] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:23.776777 12099 raft_consensus.cc:348] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.777151 12099 raft_consensus.cc:374] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:23.777284 12099 raft_consensus.cc:729] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d109cc3d55413c8fc4573f3db36192, State: Initialized, Role: FOLLOWER
I20260809 12:04:23.777499 12098 tablet_bootstrap.cc:492] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: Bootstrap starting.
I20260809 12:04:23.777643 12099 consensus_queue.cc:260] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.779294 12098 tablet_bootstrap.cc:654] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:23.780871 12099 ts_tablet_manager.cc:1428] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260809 12:04:23.789575 12098 tablet_bootstrap.cc:492] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: No bootstrap required, opened a new log
I20260809 12:04:23.789821 12098 ts_tablet_manager.cc:1397] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260809 12:04:23.790907 12098 raft_consensus.cc:348] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.791289 12098 raft_consensus.cc:374] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:23.791422 12098 raft_consensus.cc:729] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cae805193bc45509e7e7328163ce4f9, State: Initialized, Role: FOLLOWER
I20260809 12:04:23.791775 12098 consensus_queue.cc:260] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.792665 12095 tablet_bootstrap.cc:492] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: No bootstrap required, opened a new log
I20260809 12:04:23.793387 12098 ts_tablet_manager.cc:1428] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:23.792910 12095 ts_tablet_manager.cc:1397] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.004s
I20260809 12:04:23.795962 12095 raft_consensus.cc:348] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.796376 12095 raft_consensus.cc:374] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:23.796599 12095 raft_consensus.cc:729] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd8d9fdd6fd4792b96d58d568301083, State: Initialized, Role: FOLLOWER
I20260809 12:04:23.796962 12095 consensus_queue.cc:260] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:23.797714 12095 ts_tablet_manager.cc:1428] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:24.022542 12166 raft_consensus.cc:482] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:04:24.022760 12166 raft_consensus.cc:504] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.023486 12166 leader_election.cc:288] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117), f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159)
I20260809 12:04:24.024214 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "853a8907af9a412daa1e9a7a73f2fe9b" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd8d9fdd6fd4792b96d58d568301083" is_pre_election: true
I20260809 12:04:24.024255 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "853a8907af9a412daa1e9a7a73f2fe9b" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192" is_pre_election: true
I20260809 12:04:24.024464 11609 raft_consensus.cc:2454] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 0.
I20260809 12:04:24.024590 11814 raft_consensus.cc:2454] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 0.
I20260809 12:04:24.024981 11899 leader_election.cc:302] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cae805193bc45509e7e7328163ce4f9, 5bd8d9fdd6fd4792b96d58d568301083; no voters:
I20260809 12:04:24.025328 12166 raft_consensus.cc:2781] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:04:24.025473 12166 raft_consensus.cc:482] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:04:24.025591 12166 raft_consensus.cc:3037] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:24.029227 12166 raft_consensus.cc:504] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.029953 12166 leader_election.cc:288] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 election: Requested vote from peers 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117), f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159)
I20260809 12:04:24.030491 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "853a8907af9a412daa1e9a7a73f2fe9b" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192"
I20260809 12:04:24.030442 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "853a8907af9a412daa1e9a7a73f2fe9b" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd8d9fdd6fd4792b96d58d568301083"
I20260809 12:04:24.030712 11814 raft_consensus.cc:3037] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:24.030726 11609 raft_consensus.cc:3037] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:24.032774 11814 raft_consensus.cc:2454] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 1.
I20260809 12:04:24.033275 11899 leader_election.cc:302] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cae805193bc45509e7e7328163ce4f9, f9d109cc3d55413c8fc4573f3db36192; no voters:
I20260809 12:04:24.033643 12166 raft_consensus.cc:2781] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:24.033816 12166 raft_consensus.cc:686] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 1 LEADER]: Becoming Leader. State: Replica: 0cae805193bc45509e7e7328163ce4f9, State: Running, Role: LEADER
I20260809 12:04:24.034169 11609 raft_consensus.cc:2454] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 1.
I20260809 12:04:24.034089 12166 consensus_queue.cc:237] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.036798 11391 catalog_manager.cc:5495] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67). New cstate: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } health_report { overall_health: HEALTHY } } }
I20260809 12:04:24.084028 12195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:24.078734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.0.147.67:41815)
0809 12:04:24.079478 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:24.079483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:24.079498 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:04:24.079612 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:24.079616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:24.079682 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:24.079790 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:04:24.080188 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.080888 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.082251 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.082655 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.082798 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:24.083218 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:04:24.083322 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:04:24.083421 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:24.083647 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 12:04:24.083731 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":523,"threads_started":1}
I20260809 12:04:24.087242 11609 raft_consensus.cc:1261] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Refusing update from remote peer 0cae805193bc45509e7e7328163ce4f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:24.087239 11814 raft_consensus.cc:1261] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Refusing update from remote peer 0cae805193bc45509e7e7328163ce4f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:24.087980 12104 consensus_queue.cc:1035] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:24.088248 12166 consensus_queue.cc:1035] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:24.105397 11390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:24.105664 11390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:24.109148 11390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3: Sending DeleteTablet for 3 replicas of tablet 853a8907af9a412daa1e9a7a73f2fe9b
I20260809 12:04:24.110078 11779 tablet_service.cc:1514] Processing DeleteTablet for tablet 853a8907af9a412daa1e9a7a73f2fe9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:24 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:45976
I20260809 12:04:24.110230 11971 tablet_service.cc:1514] Processing DeleteTablet for tablet 853a8907af9a412daa1e9a7a73f2fe9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:24 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:33906
I20260809 12:04:24.110222 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 853a8907af9a412daa1e9a7a73f2fe9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:24 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:48568
I20260809 12:04:24.110558 12131 tablet_replica.cc:330] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: stopping tablet replica
I20260809 12:04:24.110558 12127 tablet_replica.cc:330] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: stopping tablet replica
I20260809 12:04:24.110742 12127 raft_consensus.cc:2229] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:24.110742 12131 raft_consensus.cc:2229] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:24.110973 12131 raft_consensus.cc:2258] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:24.110997 12127 raft_consensus.cc:2258] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:24.111285 12129 tablet_replica.cc:330] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: stopping tablet replica
I20260809 12:04:24.111478 12129 raft_consensus.cc:2229] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:04:24.112047 12129 raft_consensus.cc:2258] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:24.112218 12131 ts_tablet_manager.cc:1905] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:24.112562 12127 ts_tablet_manager.cc:1905] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:24.119089 12127 ts_tablet_manager.cc:1918] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:24.119217 12127 log.cc:1196] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/853a8907af9a412daa1e9a7a73f2fe9b
I20260809 12:04:24.119278 12131 ts_tablet_manager.cc:1918] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:24.119397 12131 log.cc:1196] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/853a8907af9a412daa1e9a7a73f2fe9b
I20260809 12:04:24.119866 12131 ts_tablet_manager.cc:1939] T 853a8907af9a412daa1e9a7a73f2fe9b P f9d109cc3d55413c8fc4573f3db36192: Deleting consensus metadata
I20260809 12:04:24.120587 12129 ts_tablet_manager.cc:1905] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:24.120925 11368 catalog_manager.cc:4841] TS f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159): tablet 853a8907af9a412daa1e9a7a73f2fe9b (table test-table [id=53d36dc542ab4acd82f97fa4a13b6424]) successfully deleted
I20260809 12:04:24.124264 11390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:04:24.125061 11390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:04:24.128594 12129 ts_tablet_manager.cc:1918] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:24.128772 12129 log.cc:1196] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/853a8907af9a412daa1e9a7a73f2fe9b
I20260809 12:04:24.129243 12129 ts_tablet_manager.cc:1939] T 853a8907af9a412daa1e9a7a73f2fe9b P 0cae805193bc45509e7e7328163ce4f9: Deleting consensus metadata
I20260809 12:04:24.129493 12127 ts_tablet_manager.cc:1939] T 853a8907af9a412daa1e9a7a73f2fe9b P 5bd8d9fdd6fd4792b96d58d568301083: Deleting consensus metadata
I20260809 12:04:24.130678 11374 catalog_manager.cc:4841] TS 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67:41815): tablet 853a8907af9a412daa1e9a7a73f2fe9b (table test-table [id=53d36dc542ab4acd82f97fa4a13b6424]) successfully deleted
I20260809 12:04:24.132077 11368 catalog_manager.cc:4841] TS 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117): tablet 853a8907af9a412daa1e9a7a73f2fe9b (table test-table [id=53d36dc542ab4acd82f97fa4a13b6424]) successfully deleted
I20260809 12:04:24.132357 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 3c482d64f8c747778427e0f85a49e91c (DEFAULT_TABLE table=test-table [id=da48734ce6404fa9872a4ed73ff4d7db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:24.132783 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c482d64f8c747778427e0f85a49e91c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:24.136857 11779 tablet_service.cc:1467] Processing CreateTablet for tablet 3c482d64f8c747778427e0f85a49e91c (DEFAULT_TABLE table=test-table [id=da48734ce6404fa9872a4ed73ff4d7db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:24.137012 11971 tablet_service.cc:1467] Processing CreateTablet for tablet 3c482d64f8c747778427e0f85a49e91c (DEFAULT_TABLE table=test-table [id=da48734ce6404fa9872a4ed73ff4d7db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:24.137256 11779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c482d64f8c747778427e0f85a49e91c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:24.137437 11971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c482d64f8c747778427e0f85a49e91c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:24.140048 12095 tablet_bootstrap.cc:492] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: Bootstrap starting.
I20260809 12:04:24.141719 12095 tablet_bootstrap.cc:654] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:24.143841 12099 tablet_bootstrap.cc:492] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: Bootstrap starting.
I20260809 12:04:24.145321 12095 tablet_bootstrap.cc:492] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: No bootstrap required, opened a new log
I20260809 12:04:24.145500 12099 tablet_bootstrap.cc:654] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:24.145525 12095 ts_tablet_manager.cc:1397] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 12:04:24.146275 12095 raft_consensus.cc:348] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.146620 12095 raft_consensus.cc:374] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:24.146651 12098 tablet_bootstrap.cc:492] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: Bootstrap starting.
I20260809 12:04:24.146709 12095 raft_consensus.cc:729] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd8d9fdd6fd4792b96d58d568301083, State: Initialized, Role: FOLLOWER
I20260809 12:04:24.146976 12095 consensus_queue.cc:260] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.148389 12098 tablet_bootstrap.cc:654] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:24.148535 12095 ts_tablet_manager.cc:1428] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 12:04:24.156107 12098 tablet_bootstrap.cc:492] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: No bootstrap required, opened a new log
I20260809 12:04:24.156342 12098 ts_tablet_manager.cc:1397] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 12:04:24.157204 12098 raft_consensus.cc:348] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.157486 12098 raft_consensus.cc:374] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:24.157616 12098 raft_consensus.cc:729] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cae805193bc45509e7e7328163ce4f9, State: Initialized, Role: FOLLOWER
I20260809 12:04:24.157892 12098 consensus_queue.cc:260] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.158438 12098 ts_tablet_manager.cc:1428] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 12:04:24.169981 12099 tablet_bootstrap.cc:492] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: No bootstrap required, opened a new log
I20260809 12:04:24.170188 12099 ts_tablet_manager.cc:1397] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.001s
I20260809 12:04:24.171090 12099 raft_consensus.cc:348] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.171399 12099 raft_consensus.cc:374] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:24.171485 12099 raft_consensus.cc:729] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d109cc3d55413c8fc4573f3db36192, State: Initialized, Role: FOLLOWER
I20260809 12:04:24.171780 12099 consensus_queue.cc:260] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.172278 12099 ts_tablet_manager.cc:1428] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 12:04:24.255581 12199 raft_consensus.cc:482] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:04:24.255913 12199 raft_consensus.cc:504] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.256699 12199 leader_election.cc:288] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117), f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159)
I20260809 12:04:24.257308 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c482d64f8c747778427e0f85a49e91c" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd8d9fdd6fd4792b96d58d568301083" is_pre_election: true
I20260809 12:04:24.257417 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c482d64f8c747778427e0f85a49e91c" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192" is_pre_election: true
I20260809 12:04:24.257583 11609 raft_consensus.cc:2454] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 0.
I20260809 12:04:24.257710 11814 raft_consensus.cc:2454] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 0.
I20260809 12:04:24.258116 11899 leader_election.cc:302] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cae805193bc45509e7e7328163ce4f9, 5bd8d9fdd6fd4792b96d58d568301083; no voters:
I20260809 12:04:24.258402 12199 raft_consensus.cc:2781] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:04:24.258544 12199 raft_consensus.cc:482] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:04:24.258644 12199 raft_consensus.cc:3037] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:24.262037 12199 raft_consensus.cc:504] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.262676 12199 leader_election.cc:288] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 election: Requested vote from peers 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117), f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159)
I20260809 12:04:24.263214 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c482d64f8c747778427e0f85a49e91c" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd8d9fdd6fd4792b96d58d568301083"
I20260809 12:04:24.263286 11814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c482d64f8c747778427e0f85a49e91c" candidate_uuid: "0cae805193bc45509e7e7328163ce4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d109cc3d55413c8fc4573f3db36192"
I20260809 12:04:24.263437 11609 raft_consensus.cc:3037] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:24.263487 11814 raft_consensus.cc:3037] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:24.266595 11814 raft_consensus.cc:2454] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 1.
I20260809 12:04:24.266602 11609 raft_consensus.cc:2454] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cae805193bc45509e7e7328163ce4f9 in term 1.
I20260809 12:04:24.266999 11899 leader_election.cc:302] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cae805193bc45509e7e7328163ce4f9, f9d109cc3d55413c8fc4573f3db36192; no voters:
I20260809 12:04:24.267344 12199 raft_consensus.cc:2781] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:24.267627 12199 raft_consensus.cc:686] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 1 LEADER]: Becoming Leader. State: Replica: 0cae805193bc45509e7e7328163ce4f9, State: Running, Role: LEADER
I20260809 12:04:24.267949 12199 consensus_queue.cc:237] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } }
I20260809 12:04:24.270994 11390 catalog_manager.cc:5495] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67). New cstate: current_term: 1 leader_uuid: "0cae805193bc45509e7e7328163ce4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cae805193bc45509e7e7328163ce4f9" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 41815 } health_report { overall_health: HEALTHY } } }
I20260809 12:04:24.311486 12195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:24.304611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:34603 (local address 127.0.147.67:41815)
0809 12:04:24.304802 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:24.304810 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:24.306521 (+ 1711us) server_negotiation.cc:407] Connection header received
0809 12:04:24.306742 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:24.306747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:24.306822 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:24.306922 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:04:24.307454 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.308325 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.309746 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.310187 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.310339 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:24.310743 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:04:24.310846 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:04:24.310940 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:24.311074 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 12:04:24.311141 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":57}
I20260809 12:04:24.311654 12209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:24.305284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:41815 (local address 127.0.147.67:34603)
0809 12:04:24.306401 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:24.306420 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:24.306607 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:24.306931 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:24.306935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:24.306944 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:04:24.307288 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:24.307299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.308470 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.308475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:24.309602 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:24.309613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.310349 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.310356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:24.310544 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:24.310583 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:04:24.310939 (+ 356us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:04:24.310952 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 12:04:24.311051 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 12:04:24.311179 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":6,"thread_start_us":783,"threads_started":1}
I20260809 12:04:24.320150 11814 raft_consensus.cc:1261] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Refusing update from remote peer 0cae805193bc45509e7e7328163ce4f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:24.320194 11607 raft_consensus.cc:1261] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Refusing update from remote peer 0cae805193bc45509e7e7328163ce4f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:24.320875 12199 consensus_queue.cc:1035] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bd8d9fdd6fd4792b96d58d568301083" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:24.321528 12166 consensus_queue.cc:1035] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d109cc3d55413c8fc4573f3db36192" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 38159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:24.384348 11390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:24.384630 11390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:24.388039 11390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c1f68eabc346e79a299e1f238ab1b3: Sending DeleteTablet for 3 replicas of tablet 3c482d64f8c747778427e0f85a49e91c
I20260809 12:04:24.388863 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c482d64f8c747778427e0f85a49e91c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:24 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:48568
I20260809 12:04:24.389039 11779 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c482d64f8c747778427e0f85a49e91c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:24 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:45976
I20260809 12:04:24.389202 12127 tablet_replica.cc:330] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: stopping tablet replica
I20260809 12:04:24.389346 12131 tablet_replica.cc:330] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: stopping tablet replica
I20260809 12:04:24.389413 12127 raft_consensus.cc:2229] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:24.389557 12131 raft_consensus.cc:2229] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:24.389640 12127 raft_consensus.cc:2258] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:24.389611 11971 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c482d64f8c747778427e0f85a49e91c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:24 UTC) from {username='oryx', principal='oryx/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:33906
I20260809 12:04:24.389757 12131 raft_consensus.cc:2258] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:24.389906 12129 tablet_replica.cc:330] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: stopping tablet replica
I20260809 12:04:24.390128 12129 raft_consensus.cc:2229] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:04:24.390519 12129 raft_consensus.cc:2258] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:24.390918 12127 ts_tablet_manager.cc:1905] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:24.391847 12129 ts_tablet_manager.cc:1905] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:24.392095 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 11489
I20260809 12:04:24.398387 12131 ts_tablet_manager.cc:1905] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:24.399541 12129 ts_tablet_manager.cc:1918] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:04:24.399780 12129 log.cc:1196] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/3c482d64f8c747778427e0f85a49e91c
I20260809 12:04:24.400312 12129 ts_tablet_manager.cc:1939] T 3c482d64f8c747778427e0f85a49e91c P 0cae805193bc45509e7e7328163ce4f9: Deleting consensus metadata
I20260809 12:04:24.401516 11374 catalog_manager.cc:4841] TS 0cae805193bc45509e7e7328163ce4f9 (127.0.147.67:41815): tablet 3c482d64f8c747778427e0f85a49e91c (table test-table [id=da48734ce6404fa9872a4ed73ff4d7db]) successfully deleted
I20260809 12:04:24.406010 12127 ts_tablet_manager.cc:1918] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:04:24.406203 12127 log.cc:1196] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/3c482d64f8c747778427e0f85a49e91c
I20260809 12:04:24.406656 12127 ts_tablet_manager.cc:1939] T 3c482d64f8c747778427e0f85a49e91c P 5bd8d9fdd6fd4792b96d58d568301083: Deleting consensus metadata
I20260809 12:04:24.408089 11368 catalog_manager.cc:4841] TS 5bd8d9fdd6fd4792b96d58d568301083 (127.0.147.65:38117): tablet 3c482d64f8c747778427e0f85a49e91c (table test-table [id=da48734ce6404fa9872a4ed73ff4d7db]) successfully deleted
I20260809 12:04:24.409102 12131 ts_tablet_manager.cc:1918] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:04:24.409274 12131 log.cc:1196] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/3c482d64f8c747778427e0f85a49e91c
I20260809 12:04:24.409632 12131 ts_tablet_manager.cc:1939] T 3c482d64f8c747778427e0f85a49e91c P f9d109cc3d55413c8fc4573f3db36192: Deleting consensus metadata
I20260809 12:04:24.410779 11368 catalog_manager.cc:4841] TS f9d109cc3d55413c8fc4573f3db36192 (127.0.147.66:38159): tablet 3c482d64f8c747778427e0f85a49e91c (table test-table [id=da48734ce6404fa9872a4ed73ff4d7db]) successfully deleted
Aug 09 12:04:24 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.65@KRBTEST.COM
I20260809 12:04:24.421891 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:24.403216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.0.147.65:38117)
0809 12:04:24.403809 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:24.403815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:24.403837 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:04:24.403897 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:24.403902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:24.403981 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:24.406001 (+ 2020us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:24.406233 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.407383 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.409326 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.410040 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.410179 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:24.414570 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:24.414592 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:24.414615 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:24.414657 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:24.418761 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:24.419482 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:24.419495 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:24.419500 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:24.419692 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:24.420089 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:24.420094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:24.420098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:24.420565 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:24.420881 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:24.421179 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 12:04:24.421421 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":963,"threads_started":1}
I20260809 12:04:24.422640 11672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:24.551555 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 11489
I20260809 12:04:24.585246 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 11685
Aug 09 12:04:24 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.66@KRBTEST.COM
I20260809 12:04:24.600591 12217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:24.585987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.0.147.66:38159)
0809 12:04:24.586538 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:24.586548 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:24.586564 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:04:24.586617 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:24.586621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:24.586679 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:24.586804 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:24.588133 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.588883 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.590180 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.590645 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.590783 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:24.595011 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:24.595025 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:24.595040 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:24.595081 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:24.598071 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:24.598672 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:24.598679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:24.598681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:24.598821 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:24.599225 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:24.599228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:24.599230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:24.599645 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:24.599872 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:24.600151 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 12:04:24.600317 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":245,"threads_started":1}
I20260809 12:04:24.601294 11870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:24.700222 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 11685
I20260809 12:04:24.731760 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 11879
Aug 09 12:04:24 dist-test-slave-sb2z krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.147.67@KRBTEST.COM
I20260809 12:04:24.746728 12195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:24.732523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.0.147.67:41815)
0809 12:04:24.732713 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:24.732718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:24.732738 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:04:24.732904 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:24.732908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:24.732982 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:24.733073 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:24.734461 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.735258 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.736536 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:24.736918 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:24.737019 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:24.741433 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:24.741449 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:24.741465 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:24.741497 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:24.744397 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:24.745027 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:24.745034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:24.745036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:24.745170 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:24.745495 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:24.745499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:24.745501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:24.745854 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:24.746105 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:24.746319 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 12:04:24.746480 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:04:24.747388 12058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:24.906868 12222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:24.905363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:38117 (local address 127.0.147.67:33403)
0809 12:04:24.906471 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:24.906633 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:38117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":917,"thread_start_us":776,"threads_started":1}
I20260809 12:04:24.907336 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 11879
I20260809 12:04:24.941674 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 11345
I20260809 12:04:24.942914 11446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:25.064569 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 11345
2026-08-09T12:04:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7362 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:04:25 dist-test-slave-sb2z krb5kdc[12232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:04:25 dist-test-slave-sb2z krb5kdc[12232](info): set up 2 sockets
Aug 09 12:04:25 dist-test-slave-sb2z krb5kdc[12232](info): commencing operation
krb5kdc: starting...
W20260809 12:04:26.374387 589 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:04:26 dist-test-slave-sb2z krb5kdc[12232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:04:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:04:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:04:26.662690 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38161
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:42959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:38161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:27.007189 12249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:27.007589 12249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:27.007766 12249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:27.015239 12249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:27.015390 12249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:27.015515 12249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:27.015654 12249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:27.015748 12249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:27.025900 12249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:42959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:38161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:27.027268 12249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:27.028616 12249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:27.041267 12258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:27.041282 12261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:27.041472 12264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:27.042392 12249 server_base.cc:1034] running on GCE node
I20260809 12:04:27.043223 12249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:27.044703 12249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:27.045986 12249 hybrid_clock.cc:648] HybridClock initialized: now 1786277067045909 us; error 50 us; skew 500 ppm
I20260809 12:04:27.046365 12249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T12:04:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2052 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:04:27 dist-test-slave-sb2z krb5kdc[12270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:04:27 dist-test-slave-sb2z krb5kdc[12270](info): set up 2 sockets
Aug 09 12:04:27 dist-test-slave-sb2z krb5kdc[12270](info): commencing operation
krb5kdc: starting...
W20260809 12:04:28.421206 589 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:04:28 dist-test-slave-sb2z krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:04:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:04:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:04:28.706775 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39419
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:39419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:29.043888 12285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:29.044240 12285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:29.044399 12285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:29.051587 12285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:29.051733 12285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:29.051832 12285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:29.051947 12285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:29.052032 12285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:29.061848 12285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:39419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:29.063233 12285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:29.064591 12285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:29.076386 12295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:29.076356 12297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:29.077082 12299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:29.077575 12285 server_base.cc:1034] running on GCE node
I20260809 12:04:29.078619 12285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:29.080199 12285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:29.081496 12285 hybrid_clock.cc:648] HybridClock initialized: now 1786277069081393 us; error 81 us; skew 500 ppm
I20260809 12:04:29.081859 12285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:29 dist-test-slave-sb2z krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277069, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:29.085409 12285 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:04:29.086884 12285 webserver.cc:460] Webserver started at http://127.0.147.126:34055/ using document root <none> and password file <none>
I20260809 12:04:29.087653 12285 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:29.087810 12285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:29.088124 12285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:29.091135 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "bbed5e3d061d4110bfb31dba5be0e807"
format_stamp: "Formatted at 2026-08-09 12:04:29 on dist-test-slave-sb2z"
server_key: "3de77a25a8f57eef7f006ee1b5933fb3"
server_key_iv: "5600239b58f03d0ffc8dd16ad92cb04c"
server_key_version: "encryptionkey@0"
I20260809 12:04:29.091974 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "bbed5e3d061d4110bfb31dba5be0e807"
format_stamp: "Formatted at 2026-08-09 12:04:29 on dist-test-slave-sb2z"
server_key: "3de77a25a8f57eef7f006ee1b5933fb3"
server_key_iv: "5600239b58f03d0ffc8dd16ad92cb04c"
server_key_version: "encryptionkey@0"
I20260809 12:04:29.097000 12285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:04:29.100455 12308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:29.101799 12285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:04:29.102005 12285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "bbed5e3d061d4110bfb31dba5be0e807"
format_stamp: "Formatted at 2026-08-09 12:04:29 on dist-test-slave-sb2z"
server_key: "3de77a25a8f57eef7f006ee1b5933fb3"
server_key_iv: "5600239b58f03d0ffc8dd16ad92cb04c"
server_key_version: "encryptionkey@0"
I20260809 12:04:29.102206 12285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:29.113309 12285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:29.117610 12285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:29.117902 12285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:29.137065 12285 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:39419
I20260809 12:04:29.137100 12375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:39419 every 8 connection(s)
I20260809 12:04:29.138458 12285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:04:29.142220 12378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:29.144140 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 12285
I20260809 12:04:29.144366 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:04:29.145049 589 external_mini_cluster.cc:1428] Setting key 17cd500f82df54c5552a44cb9fb91599
I20260809 12:04:29.154556 12378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: Bootstrap starting.
I20260809 12:04:29.158598 12378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:29.159875 12378 log.cc:824] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:04:29 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:29.163710 12378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: No bootstrap required, opened a new log
I20260809 12:04:29.168776 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:29.147017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50252 (local address 127.0.147.126:39419)
0809 12:04:29.147915 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:29.147929 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:29.147974 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:04:29.149615 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:29.149643 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:29.150016 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:29.150316 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:29.152318 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:29.153604 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:29.155396 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:29.156117 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:29.156258 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:29.161480 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:29.161492 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:29.161506 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:29.161546 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:29.164734 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:29.165454 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:29.165469 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:29.165479 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:29.165653 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:29.166050 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:29.166055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:29.166065 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:29.166596 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:29.166929 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:29.167370 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 12:04:29.167823 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":442,"threads_started":1}
I20260809 12:04:29.171830 12378 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } }
I20260809 12:04:29.172183 12378 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:29.172302 12378 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbed5e3d061d4110bfb31dba5be0e807, State: Initialized, Role: FOLLOWER
I20260809 12:04:29.172887 12378 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } }
I20260809 12:04:29.173187 12378 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:04:29.173382 12378 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:04:29.173588 12378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:29.175993 12378 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } }
I20260809 12:04:29.176558 12378 leader_election.cc:302] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbed5e3d061d4110bfb31dba5be0e807; no voters:
I20260809 12:04:29.177171 12378 leader_election.cc:288] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:04:29.177286 12383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:29.178145 12383 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 1 LEADER]: Becoming Leader. State: Replica: bbed5e3d061d4110bfb31dba5be0e807, State: Running, Role: LEADER
I20260809 12:04:29.178727 12383 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } }
I20260809 12:04:29.179407 12378 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:29.181586 12384 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbed5e3d061d4110bfb31dba5be0e807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } } }
I20260809 12:04:29.182055 12384 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:29.181589 12385 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbed5e3d061d4110bfb31dba5be0e807. Latest consensus state: current_term: 1 leader_uuid: "bbed5e3d061d4110bfb31dba5be0e807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } } }
I20260809 12:04:29.183789 12385 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:29.185822 12390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:29.191077 12390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:29.199779 12390 catalog_manager.cc:1347] Generated new cluster ID: 7cec4c5855b74d32a1d779b52d2ae1e7
I20260809 12:04:29.199909 12390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:29.211141 12390 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:04:29.212659 12390 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:29.220239 12390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: Generated new TSK 0
I20260809 12:04:29.221042 12390 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:04:29.297233 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39419
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:29.663913 12413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:29.664194 12413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:29.664289 12413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:29.671662 12413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:29.671797 12413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:29.671949 12413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:04:29.681754 12413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:29.682991 12413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:29.684567 12413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:29.697540 12421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:29.697700 12422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:29.698001 12425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:29.698951 12413 server_base.cc:1034] running on GCE node
I20260809 12:04:29.699632 12413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:29.700559 12413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:29.701828 12413 hybrid_clock.cc:648] HybridClock initialized: now 1786277069701774 us; error 50 us; skew 500 ppm
I20260809 12:04:29.702208 12413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:29 dist-test-slave-sb2z krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277069, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:29.705567 12413 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:04:29.707118 12413 webserver.cc:460] Webserver started at http://127.0.147.65:34169/ using document root <none> and password file <none>
I20260809 12:04:29.707912 12413 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:29.708066 12413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:29.708380 12413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:29.711256 12413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "854ce2fbf31c4f9aa2c3cb798064a204"
format_stamp: "Formatted at 2026-08-09 12:04:29 on dist-test-slave-sb2z"
server_key: "4a442e30aa03e3da07148329122d23ec"
server_key_iv: "0fb83f782e08af317729919d3c499ac0"
server_key_version: "encryptionkey@0"
I20260809 12:04:29.712131 12413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "854ce2fbf31c4f9aa2c3cb798064a204"
format_stamp: "Formatted at 2026-08-09 12:04:29 on dist-test-slave-sb2z"
server_key: "4a442e30aa03e3da07148329122d23ec"
server_key_iv: "0fb83f782e08af317729919d3c499ac0"
server_key_version: "encryptionkey@0"
I20260809 12:04:29.717170 12413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:04:29.720775 12432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:29.722124 12413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:04:29.722291 12413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "854ce2fbf31c4f9aa2c3cb798064a204"
format_stamp: "Formatted at 2026-08-09 12:04:29 on dist-test-slave-sb2z"
server_key: "4a442e30aa03e3da07148329122d23ec"
server_key_iv: "0fb83f782e08af317729919d3c499ac0"
server_key_version: "encryptionkey@0"
I20260809 12:04:29.722484 12413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:29.753880 12413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:29.758260 12413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:29.758514 12413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:29.759460 12413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:29.760851 12413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:29.760941 12413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:29.761035 12413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:29.761082 12413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:29.796577 12413 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:39915
I20260809 12:04:29.796628 12593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:39915 every 8 connection(s)
I20260809 12:04:29.797931 12413 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:04:29.799388 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 12413
I20260809 12:04:29.799633 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:04:29.800454 589 external_mini_cluster.cc:1428] Setting key 606e041a8029c9f02d3ea903380709c6
Aug 09 12:04:29 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277069, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:29.820062 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:29.800940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:60717 (local address 127.0.147.126:39419)
0809 12:04:29.801489 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:29.801496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:29.802676 (+ 1180us) server_negotiation.cc:407] Connection header received
0809 12:04:29.804616 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:29.804621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:29.804728 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:29.804872 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:29.806476 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:29.807334 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:29.808934 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:29.809334 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:29.809485 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:29.813367 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:29.813373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:29.813376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:29.813409 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:29.816542 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:29.817552 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:29.817557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:29.817559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:29.817687 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:29.818203 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:29.818206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:29.818208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:29.818482 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:29.818601 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:29.819595 (+ 994us) server_negotiation.cc:299] Negotiation successful
0809 12:04:29.819788 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":261,"threads_started":1}
I20260809 12:04:29.821689 12598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:29.801562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:60717)
0809 12:04:29.802461 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:29.802525 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:29.804351 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:29.805113 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:29.805121 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:29.805722 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:29.806298 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:29.806315 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:29.807505 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:29.807509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:29.808781 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:29.808793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:29.809588 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:29.809598 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:29.809832 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:29.810545 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:29.810566 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:29.813111 (+ 2545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:29.816693 (+ 3582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:29.816706 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:29.816728 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:29.817405 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:29.817803 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:29.817810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:29.817816 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:29.818085 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:29.818611 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:29.818622 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:29.819102 (+ 480us) client_negotiation.cc:769] Sending connection context
0809 12:04:29.819423 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 12:04:29.820156 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":401,"threads_started":1}
I20260809 12:04:29.823460 12594 heartbeater.cc:347] Connected to a master server at 127.0.147.126:39419
I20260809 12:04:29.823848 12594 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:29.824618 12594 heartbeater.cc:511] Master 127.0.147.126:39419 requested a full tablet report, sending...
I20260809 12:04:29.826999 12331 ts_manager.cc:194] Registered new tserver with Master: 854ce2fbf31c4f9aa2c3cb798064a204 (127.0.147.65:39915)
I20260809 12:04:29.828572 12331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:60717
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:04:29.868489 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39419
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:30.208037 12602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:30.208297 12602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:30.208470 12602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:30.215700 12602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:30.215796 12602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:30.215989 12602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:04:30.225553 12602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:30.226867 12602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:30.228163 12602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:30.241254 12611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:30.241288 12609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:30.241220 12608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:30.242344 12602 server_base.cc:1034] running on GCE node
I20260809 12:04:30.242923 12602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:30.243907 12602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:30.245139 12602 hybrid_clock.cc:648] HybridClock initialized: now 1786277070245099 us; error 52 us; skew 500 ppm
I20260809 12:04:30.245443 12602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:30 dist-test-slave-sb2z krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277070, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:30.248586 12602 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:04:30.250003 12602 webserver.cc:460] Webserver started at http://127.0.147.66:43925/ using document root <none> and password file <none>
I20260809 12:04:30.250679 12602 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:30.250825 12602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:30.251152 12602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:30.254130 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "0219ec33656a4cb88ac35a70d7217e09"
format_stamp: "Formatted at 2026-08-09 12:04:30 on dist-test-slave-sb2z"
server_key: "65bd182eeb49a340e38f7d12dbb884f5"
server_key_iv: "de4a2c0ec2dd15794f5f43d28b52a475"
server_key_version: "encryptionkey@0"
I20260809 12:04:30.254985 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "0219ec33656a4cb88ac35a70d7217e09"
format_stamp: "Formatted at 2026-08-09 12:04:30 on dist-test-slave-sb2z"
server_key: "65bd182eeb49a340e38f7d12dbb884f5"
server_key_iv: "de4a2c0ec2dd15794f5f43d28b52a475"
server_key_version: "encryptionkey@0"
I20260809 12:04:30.260077 12602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:04:30.263446 12622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:30.264755 12602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:04:30.264930 12602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "0219ec33656a4cb88ac35a70d7217e09"
format_stamp: "Formatted at 2026-08-09 12:04:30 on dist-test-slave-sb2z"
server_key: "65bd182eeb49a340e38f7d12dbb884f5"
server_key_iv: "de4a2c0ec2dd15794f5f43d28b52a475"
server_key_version: "encryptionkey@0"
I20260809 12:04:30.265159 12602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:30.282519 12602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:30.287024 12602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:30.287273 12602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:30.288355 12602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:30.289669 12602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:30.289750 12602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:30.289839 12602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:30.289883 12602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:30.324277 12602 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:37479
I20260809 12:04:30.324363 12782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:37479 every 8 connection(s)
I20260809 12:04:30.325510 12602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:04:30.326517 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 12602
I20260809 12:04:30.326732 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:04:30.327299 589 external_mini_cluster.cc:1428] Setting key 4f973204c163896ac9a55738f192aedf
Aug 09 12:04:30 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277070, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:30.347193 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:30.328520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:41951 (local address 127.0.147.126:39419)
0809 12:04:30.328982 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:30.328988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:30.330064 (+ 1076us) server_negotiation.cc:407] Connection header received
0809 12:04:30.331975 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:30.331981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:30.332089 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:30.332204 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:30.333690 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:30.334446 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:30.336051 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:30.336480 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:30.336615 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:30.340557 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:30.340562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:30.340565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:30.340597 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:30.343925 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:30.344590 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:30.344597 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:30.344603 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:30.344758 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:30.345194 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:30.345202 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:30.345208 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:30.345617 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:30.345773 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:30.346398 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 12:04:30.346649 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":242,"threads_started":1}
I20260809 12:04:30.347677 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:30.329047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:41951)
0809 12:04:30.329891 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:30.329935 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:30.331713 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:30.332454 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:30.332462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:30.332980 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:30.333523 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:30.333540 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:30.334623 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:30.334627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:30.335902 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:30.335914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:30.336723 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:30.336731 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:30.336952 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:30.337635 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:30.337661 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:30.340332 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:30.344056 (+ 3724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:30.344062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:30.344072 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:30.344459 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:30.344864 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:30.344867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:30.344870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:30.345058 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:30.345738 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:30.345744 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:30.346028 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 12:04:30.346246 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 12:04:30.346583 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":381,"threads_started":1}
I20260809 12:04:30.349159 12783 heartbeater.cc:347] Connected to a master server at 127.0.147.126:39419
I20260809 12:04:30.349471 12783 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:30.350334 12783 heartbeater.cc:511] Master 127.0.147.126:39419 requested a full tablet report, sending...
I20260809 12:04:30.352078 12331 ts_manager.cc:194] Registered new tserver with Master: 0219ec33656a4cb88ac35a70d7217e09 (127.0.147.66:37479)
I20260809 12:04:30.353371 12331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:41951
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:04:30.398178 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39419
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:30.734835 12790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:30.735121 12790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:30.735296 12790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:30.742359 12790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:30.742460 12790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:30.742632 12790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:04:30.752214 12790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:39419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:30.753513 12790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:30.754846 12790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:30.768664 12801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:30.768805 12804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:30.768541 12802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:30.769554 12790 server_base.cc:1034] running on GCE node
I20260809 12:04:30.770188 12790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:30.771163 12790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:30.772408 12790 hybrid_clock.cc:648] HybridClock initialized: now 1786277070772360 us; error 37 us; skew 500 ppm
I20260809 12:04:30.772791 12790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:30 dist-test-slave-sb2z krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277070, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:30.776062 12790 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:04:30.777557 12790 webserver.cc:460] Webserver started at http://127.0.147.67:37471/ using document root <none> and password file <none>
I20260809 12:04:30.778245 12790 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:30.778371 12790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:30.778726 12790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:30.781878 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "7a2baf0f6e1e48da9a7a6413e3500e57"
format_stamp: "Formatted at 2026-08-09 12:04:30 on dist-test-slave-sb2z"
server_key: "2da302160826c188cd5b6d5de106d016"
server_key_iv: "8155fcb955a0ac79fae1f4a461bd080c"
server_key_version: "encryptionkey@0"
I20260809 12:04:30.782684 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "7a2baf0f6e1e48da9a7a6413e3500e57"
format_stamp: "Formatted at 2026-08-09 12:04:30 on dist-test-slave-sb2z"
server_key: "2da302160826c188cd5b6d5de106d016"
server_key_iv: "8155fcb955a0ac79fae1f4a461bd080c"
server_key_version: "encryptionkey@0"
I20260809 12:04:30.787859 12790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:04:30.791214 12813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:30.792526 12790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:04:30.792698 12790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "7a2baf0f6e1e48da9a7a6413e3500e57"
format_stamp: "Formatted at 2026-08-09 12:04:30 on dist-test-slave-sb2z"
server_key: "2da302160826c188cd5b6d5de106d016"
server_key_iv: "8155fcb955a0ac79fae1f4a461bd080c"
server_key_version: "encryptionkey@0"
I20260809 12:04:30.792901 12790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:30.815922 12790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:30.820214 12790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:30.820511 12790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:30.821509 12790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:30.822836 12790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:30.822943 12790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:30.823108 12790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:30.823186 12790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:30.831629 12594 heartbeater.cc:503] Master 127.0.147.126:39419 was elected leader, sending a full tablet report...
I20260809 12:04:30.858260 12790 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:42997
I20260809 12:04:30.858321 12969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:42997 every 8 connection(s)
I20260809 12:04:30.859681 12790 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:04:30.865664 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 12790
I20260809 12:04:30.865861 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:04:30.866323 589 external_mini_cluster.cc:1428] Setting key 0789283c220ceba2e7714777cb2cfa3c
Aug 09 12:04:30 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277070, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:30.881661 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:30.862284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:47973 (local address 127.0.147.126:39419)
0809 12:04:30.862770 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:30.862775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:30.863681 (+ 906us) server_negotiation.cc:407] Connection header received
0809 12:04:30.865662 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:30.865667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:30.865733 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:30.865892 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:30.867670 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:30.868786 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:30.870506 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:30.870891 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:30.871021 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:30.874493 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:30.874497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:30.874500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:30.874532 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:30.877670 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:30.878624 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:30.878635 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:30.878639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:30.878833 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:30.879407 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:30.879412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:30.879415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:30.879867 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:30.880088 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:30.880944 (+ 856us) server_negotiation.cc:299] Negotiation successful
0809 12:04:30.881206 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":270,"threads_started":1}
I20260809 12:04:30.882740 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:30.862777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:47973)
0809 12:04:30.863477 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:30.863513 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:30.865353 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:30.866136 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:30.866144 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:30.866712 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:30.867461 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:30.867487 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:30.868975 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:30.868980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:30.870351 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:30.870363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:30.871116 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:30.871123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:30.871332 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:30.872034 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:30.872056 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:30.874289 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:30.877834 (+ 3545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:30.877845 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:30.877860 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:30.878489 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:30.878976 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:30.878981 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:30.878986 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:30.879244 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:30.880009 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:30.880017 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:30.880423 (+ 406us) client_negotiation.cc:769] Sending connection context
0809 12:04:30.880726 (+ 303us) client_negotiation.cc:240] Negotiation successful
0809 12:04:30.881291 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":287,"threads_started":1}
I20260809 12:04:30.885257 12970 heartbeater.cc:347] Connected to a master server at 127.0.147.126:39419
I20260809 12:04:30.885653 12970 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:30.886725 12970 heartbeater.cc:511] Master 127.0.147.126:39419 requested a full tablet report, sending...
I20260809 12:04:30.888629 12331 ts_manager.cc:194] Registered new tserver with Master: 7a2baf0f6e1e48da9a7a6413e3500e57 (127.0.147.67:42997)
I20260809 12:04:30.889814 12331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:47973
I20260809 12:04:30.894001 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab.
Entry for principal oryx/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab.
WARNING: no policy specified for oryx/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab.
Entry for principal oryx/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab.
WARNING: no policy specified for oryx/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab.
Entry for principal oryx/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab.
I20260809 12:04:31.062692 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 12413
Aug 09 12:04:31 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:04:31.079104 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:31.063802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.0.147.65:39915)
0809 12:04:31.064786 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:31.064794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:31.064823 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:04:31.064886 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:31.064892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:31.065124 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:31.065264 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:31.066548 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:31.067608 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:31.068889 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:31.069523 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:31.069618 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:31.073766 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:31.073778 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:31.073792 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:31.073830 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:31.076618 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:31.077225 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:31.077232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:31.077234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:31.077356 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:31.077701 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:31.077704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:31.077706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:31.078032 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:31.078256 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:31.078484 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 12:04:31.078702 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":652,"threads_started":1}
I20260809 12:04:31.080621 12588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:31.166010 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 12413
I20260809 12:04:31.189846 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 12602
Aug 09 12:04:31 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:04:31.206254 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:31.190868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.0.147.66:37479)
0809 12:04:31.191412 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:31.191419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:31.191445 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:04:31.191519 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:31.191574 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:31.191789 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:31.191920 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:31.193151 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:31.194156 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:31.195375 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:31.195987 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:31.196085 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:31.200470 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:31.200482 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:31.200496 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:31.200537 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:31.203668 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:31.204304 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:31.204311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:31.204313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:31.204437 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:31.204750 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:31.204753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:31.204755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:31.205083 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:31.205328 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:31.205648 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 12:04:31.205852 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":267,"threads_started":1}
I20260809 12:04:31.207701 12777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:31.292757 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 12602
I20260809 12:04:31.320039 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 12790
Aug 09 12:04:31 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:04:31.338661 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:31.321426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56658 (local address 127.0.147.67:42997)
0809 12:04:31.322181 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:31.322189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:31.322220 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:04:31.322285 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:31.322291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:31.322555 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:31.322730 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:31.324609 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:31.326223 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:31.327823 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:31.328389 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:31.328543 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:31.332957 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:31.332968 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:31.332982 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:31.333025 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:31.335820 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:31.336519 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:31.336526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:31.336529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:31.336660 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:31.337074 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:31.337078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:31.337080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:31.337413 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:31.337729 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:31.338039 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 12:04:31.338308 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":317,"threads_started":1}
I20260809 12:04:31.339983 12959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:31.425343 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 12790
I20260809 12:04:31.444739 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 12285
I20260809 12:04:31.445952 12370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:31.552819 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 12285
I20260809 12:04:31.580497 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39419
--webserver_interface=127.0.147.126
--webserver_port=34055
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:39419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:31.919014 12994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:31.919358 12994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:31.919564 12994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:31.926698 12994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:31.926841 12994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:31.926964 12994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:31.927078 12994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:31.927160 12994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:31.936708 12994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:39419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=34055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:31.937985 12994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:31.939280 12994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:31.951152 13002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:31.951226 12999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:31.951195 13000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:31.952204 12994 server_base.cc:1034] running on GCE node
I20260809 12:04:31.952955 12994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:31.954226 12994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:31.955497 12994 hybrid_clock.cc:648] HybridClock initialized: now 1786277071955463 us; error 55 us; skew 500 ppm
I20260809 12:04:31.955930 12994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:31 dist-test-slave-sb2z krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277071, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:31.959165 12994 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:04:31.960623 12994 webserver.cc:460] Webserver started at http://127.0.147.126:34055/ using document root <none> and password file <none>
I20260809 12:04:31.961325 12994 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:31.961503 12994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:31.966764 12994 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 12:04:31.969718 13010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:31.971094 12994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:31.971235 12994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "bbed5e3d061d4110bfb31dba5be0e807"
format_stamp: "Formatted at 2026-08-09 12:04:29 on dist-test-slave-sb2z"
server_key: "3de77a25a8f57eef7f006ee1b5933fb3"
server_key_iv: "5600239b58f03d0ffc8dd16ad92cb04c"
server_key_version: "encryptionkey@0"
I20260809 12:04:31.972226 12994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:31.993431 12994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:31.997299 12994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:31.997622 12994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:32.017886 12994 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:39419
I20260809 12:04:32.017956 13075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:39419 every 8 connection(s)
I20260809 12:04:32.019435 12994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:04:32.026360 13076 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:04:32.028561 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 12994
I20260809 12:04:32.029385 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:39915
--local_ip_for_outbound_sockets=127.0.147.65
--tserver_master_addrs=127.0.147.126:39419
--webserver_port=34169
--webserver_interface=127.0.147.65
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.147.65
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:04:32.029688 13076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: Bootstrap starting.
I20260809 12:04:32.051460 13076 log.cc:824] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:32.064602 13076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:04:32.065363 13076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: Bootstrap complete.
I20260809 12:04:32.080226 13076 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } }
I20260809 12:04:32.081477 13076 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbed5e3d061d4110bfb31dba5be0e807, State: Initialized, Role: FOLLOWER
I20260809 12:04:32.082221 13076 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } }
I20260809 12:04:32.082504 13076 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:04:32.082634 13076 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:04:32.082813 13076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:04:32.088393 13076 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } }
I20260809 12:04:32.089043 13076 leader_election.cc:302] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbed5e3d061d4110bfb31dba5be0e807; no voters:
I20260809 12:04:32.090013 13076 leader_election.cc:288] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:04:32.090209 13080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:04:32.091646 13080 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [term 2 LEADER]: Becoming Leader. State: Replica: bbed5e3d061d4110bfb31dba5be0e807, State: Running, Role: LEADER
I20260809 12:04:32.092252 13076 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:32.092176 13080 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } }
I20260809 12:04:32.094722 13082 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbed5e3d061d4110bfb31dba5be0e807. Latest consensus state: current_term: 2 leader_uuid: "bbed5e3d061d4110bfb31dba5be0e807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } } }
I20260809 12:04:32.094816 13081 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bbed5e3d061d4110bfb31dba5be0e807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbed5e3d061d4110bfb31dba5be0e807" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39419 } } }
I20260809 12:04:32.095112 13081 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:32.095108 13082 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:32.109619 13085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:32.114943 13085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:32.116423 13085 catalog_manager.cc:1259] Loaded cluster ID: 7cec4c5855b74d32a1d779b52d2ae1e7
I20260809 12:04:32.116523 13085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:32.120611 13085 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:32.122048 13085 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bbed5e3d061d4110bfb31dba5be0e807: Loaded TSK: 0
I20260809 12:04:32.123497 13085 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:04:32.408941 13078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:32.409264 13078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:32.409446 13078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:32.416762 13078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:32.416915 13078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:32.417109 13078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:04:32.427492 13078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.65.keytab
--principal=oryx/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:39915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=34169
--tserver_master_addrs=127.0.147.126:39419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:32.428867 13078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:32.430204 13078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:32.444794 13112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:32.444973 13115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:32.445036 13111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:32.446659 13078 server_base.cc:1034] running on GCE node
I20260809 12:04:32.447336 13078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:32.448406 13078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:32.449638 13078 hybrid_clock.cc:648] HybridClock initialized: now 1786277072449594 us; error 39 us; skew 500 ppm
I20260809 12:04:32.449957 13078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:32 dist-test-slave-sb2z krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277072, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:32.457381 13078 init.cc:377] Logged in from keytab as oryx/127.0.147.65@KRBTEST.COM (short username oryx)
I20260809 12:04:32.459025 13078 webserver.cc:460] Webserver started at http://127.0.147.65:34169/ using document root <none> and password file <none>
I20260809 12:04:32.460116 13078 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:32.460309 13078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:32.465899 13078 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 12:04:32.468616 13127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:32.469883 13078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:04:32.470075 13078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "854ce2fbf31c4f9aa2c3cb798064a204"
format_stamp: "Formatted at 2026-08-09 12:04:29 on dist-test-slave-sb2z"
server_key: "4a442e30aa03e3da07148329122d23ec"
server_key_iv: "0fb83f782e08af317729919d3c499ac0"
server_key_version: "encryptionkey@0"
I20260809 12:04:32.471074 13078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:32.496209 13078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:32.500939 13078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:32.501247 13078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:32.502243 13078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:32.503615 13078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:32.503748 13078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:32.503873 13078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:32.503966 13078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:32.538066 13078 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:39915
I20260809 12:04:32.538153 13281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:39915 every 8 connection(s)
I20260809 12:04:32.539570 13078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:04:32.544924 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 13078
I20260809 12:04:32.545554 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:37479
--local_ip_for_outbound_sockets=127.0.147.66
--tserver_master_addrs=127.0.147.126:39419
--webserver_port=43925
--webserver_interface=127.0.147.66
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.147.66
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:04:32 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:32 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:04:32.893603 13288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:32.893921 13288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:32.894076 13288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:32.901322 13288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:32.901499 13288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:32.901672 13288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:04:32.911481 13288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.66.keytab
--principal=oryx/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:37479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=43925
--tserver_master_addrs=127.0.147.126:39419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:32.912886 13288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:32.914181 13288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:32.932657 13296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:32.932606 13297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:32.932837 13300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:32.935218 13288 server_base.cc:1034] running on GCE node
I20260809 12:04:32.936045 13288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:32.937148 13288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:32.938544 13288 hybrid_clock.cc:648] HybridClock initialized: now 1786277072938468 us; error 75 us; skew 500 ppm
I20260809 12:04:32.938885 13288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:32 dist-test-slave-sb2z krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277072, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:32.942569 13288 init.cc:377] Logged in from keytab as oryx/127.0.147.66@KRBTEST.COM (short username oryx)
I20260809 12:04:32.944411 13288 webserver.cc:460] Webserver started at http://127.0.147.66:43925/ using document root <none> and password file <none>
I20260809 12:04:32.945151 13288 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:32.945319 13288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:32.950932 13288 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 12:04:32.953815 13312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:32.955229 13288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:32.955405 13288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "0219ec33656a4cb88ac35a70d7217e09"
format_stamp: "Formatted at 2026-08-09 12:04:30 on dist-test-slave-sb2z"
server_key: "65bd182eeb49a340e38f7d12dbb884f5"
server_key_iv: "de4a2c0ec2dd15794f5f43d28b52a475"
server_key_version: "encryptionkey@0"
I20260809 12:04:32.956416 13288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:32.987674 13288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:32.992352 13288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:32.992645 13288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:32.993769 13288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:32.995254 13288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:32.995368 13288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:32.995493 13288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:32.995647 13288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 12:04:33.046383 13288 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:37479
I20260809 12:04:33.046437 13452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:37479 every 8 connection(s)
I20260809 12:04:33.047881 13288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:04:33.049991 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 13288
I20260809 12:04:33.050674 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:42997
--local_ip_for_outbound_sockets=127.0.147.67
--tserver_master_addrs=127.0.147.126:39419
--webserver_port=37471
--webserver_interface=127.0.147.67
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.147.67
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:04:33 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:33 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:04:33.442510 13462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:33.443107 13462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:33.443359 13462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:33.462311 13462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:33.462563 13462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:33.462872 13462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:04:33.488549 13462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/krb5kdc/oryx_127.0.147.67.keytab
--principal=oryx/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:42997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=37471
--tserver_master_addrs=127.0.147.126:39419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:33.491114 13462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:33.493638 13462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:33.524366 13468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:33.529201 13469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:33.530189 13462 server_base.cc:1034] running on GCE node
W20260809 12:04:33.530524 13472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:33.531801 13462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:33.533627 13462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:33.535331 13462 hybrid_clock.cc:648] HybridClock initialized: now 1786277073535234 us; error 69 us; skew 500 ppm
I20260809 12:04:33.536082 13462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:33 dist-test-slave-sb2z krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277073, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:33.541939 13462 init.cc:377] Logged in from keytab as oryx/127.0.147.67@KRBTEST.COM (short username oryx)
I20260809 12:04:33.553936 13462 webserver.cc:460] Webserver started at http://127.0.147.67:37471/ using document root <none> and password file <none>
I20260809 12:04:33.555380 13462 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:33.555683 13462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:33.566699 13462 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.002s
I20260809 12:04:33.572064 13480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:33.574630 13462 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260809 12:04:33.574965 13462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "7a2baf0f6e1e48da9a7a6413e3500e57"
format_stamp: "Formatted at 2026-08-09 12:04:30 on dist-test-slave-sb2z"
server_key: "2da302160826c188cd5b6d5de106d016"
server_key_iv: "8155fcb955a0ac79fae1f4a461bd080c"
server_key_version: "encryptionkey@0"
I20260809 12:04:33.577533 13462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:33.634328 13462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:33.643034 13462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:33.643632 13462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:33.645556 13462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:33.648324 13462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:33.648574 13462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 12:04:33.648803 13462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:33.648977 13462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:33.716341 13462 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:42997
I20260809 12:04:33.716459 13637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:42997 every 8 connection(s)
I20260809 12:04:33.719158 13462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:04:33.729318 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 13462
Aug 09 12:04:33 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:33 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:33.764050 13644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:33.731200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.0.147.126:39419)
0809 12:04:33.740542 (+ 9342us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:33.740547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:33.741562 (+ 1015us) server_negotiation.cc:407] Connection header received
0809 12:04:33.741716 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:33.741725 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:33.741836 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:33.741984 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:33.744374 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.745543 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.747769 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.748540 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.748684 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:33.752767 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:33.752779 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:33.752801 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:33.752846 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:33.757517 (+ 4671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:33.758540 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:33.758554 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:33.758580 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:33.758887 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:33.759372 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:33.759381 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:33.759387 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:33.760131 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:33.760485 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:33.762052 (+ 1567us) server_negotiation.cc:299] Negotiation successful
0809 12:04:33.762582 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9211,"thread_start_us":9080,"threads_started":1}
W20260809 12:04:34.006640 13286 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.454s user 0.002s sys 0.017s
W20260809 12:04:34.008994 13285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:32.542591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:44449 (local address 127.0.147.126:39419)
0809 12:04:32.543417 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:32.543435 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:32.543716 (+ 281us) server_negotiation.cc:407] Connection header received
0809 12:04:32.546016 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:32.546031 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:32.546354 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:32.546694 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:32.548255 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:32.549496 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:32.551038 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:32.551749 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:32.551967 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.008476 (+1456509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:44449: BlockingRecv error: recv got EOF from 127.0.147.65:44449 (error 108)
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":295,"threads_started":1}
I20260809 12:04:34.011225 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:32.542636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:44449)
0809 12:04:32.543481 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:32.543593 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:32.545413 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:32.546847 (+ 1434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:32.546857 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:32.547465 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:32.548050 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:32.548067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:32.549667 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:32.549672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:32.550886 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:32.550898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:32.552006 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:32.552014 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:32.552286 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:32.552980 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:32.553002 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:34.009467 (+1456465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":354,"threads_started":1}
W20260809 12:04:34.012981 13282 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:39419 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.147.126:39419: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:04:34.034109 13285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:34.014060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:34343 (local address 127.0.147.126:39419)
0809 12:04:34.014335 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:34.014345 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:34.014513 (+ 168us) server_negotiation.cc:407] Connection header received
0809 12:04:34.014751 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:34.014759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:34.014877 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:34.015038 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:34.016396 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.018077 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.023602 (+ 5525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.024845 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.025057 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.033700 (+ 8643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:34343: BlockingRecv error: recv got EOF from 127.0.147.65:34343 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 12:04:34.038142 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:34.013935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:34343)
0809 12:04:34.014358 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:34.014388 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:34.014586 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:34.015100 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:34.015108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:34.015846 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:34.016146 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.016164 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.018384 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.018394 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.020744 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.020765 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.025514 (+ 4749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.025524 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.025828 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.026916 (+ 1088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:34.026955 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:34.037625 (+ 10670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233}
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:04:34.059891 13285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:34.040005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:54709 (local address 127.0.147.126:39419)
0809 12:04:34.040278 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:34.040288 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:34.040350 (+ 62us) server_negotiation.cc:407] Connection header received
0809 12:04:34.040430 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:34.040439 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:34.040568 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:34.040721 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:34.041956 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.043760 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.046760 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.047972 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.048127 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.059336 (+ 11209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:54709: BlockingRecv error: recv got EOF from 127.0.147.65:54709 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 12:04:34.060997 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:34.039714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:54709)
0809 12:04:34.040058 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:34.040084 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:34.040227 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:34.040799 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:34.040807 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:34.041444 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:34.041739 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.041755 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.044013 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.044021 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.046489 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.046515 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.051470 (+ 4955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.051482 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.051845 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.052915 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:34.052953 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:34.060568 (+ 7615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186}
W20260809 12:04:34.061585 13282 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:04:34.379176 13459 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.301s user 0.007s sys 0.014s
W20260809 12:04:34.380587 13457 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:33.051243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:48127 (local address 127.0.147.126:39419)
0809 12:04:33.067962 (+ 16719us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:33.067966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:33.067985 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:04:33.068040 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:33.068043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:33.068115 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:33.068261 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:33.069910 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.071248 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.074915 (+ 3667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.075905 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.076081 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.380248 (+1304167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:48127: BlockingRecv error: recv got EOF from 127.0.147.66:48127 (error 108)
Metrics: {"server-negotiator.queue_time_us":16586,"thread_start_us":16452,"threads_started":1}
I20260809 12:04:34.381271 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:33.052284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:48127)
0809 12:04:33.064086 (+ 11802us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:33.064150 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:33.067061 (+ 2911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:33.068546 (+ 1485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:33.068555 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:33.069119 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:33.069701 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:33.069719 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.071477 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.071487 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:33.074703 (+ 3216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:33.074731 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.076285 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.076296 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:33.076892 (+ 596us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:33.077994 (+ 1102us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:33.078040 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:34.380065 (+1302025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":11394,"thread_start_us":10974,"threads_started":1}
W20260809 12:04:34.382405 13453 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:39419 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.147.126:39419: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:34.393497 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:34.383319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:59075)
0809 12:04:34.383555 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:34.383575 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:34.383725 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:34.384075 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:34.384080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:34.384504 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:34.384672 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.384681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.385890 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.385896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.386866 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.386878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.387710 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.387717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.387895 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.388743 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:34.388761 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:34.393201 (+ 4440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260809 12:04:34.393621 13457 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:34.383283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:59075 (local address 127.0.147.126:39419)
0809 12:04:34.383448 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:34.383454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:34.383656 (+ 202us) server_negotiation.cc:407] Connection header received
0809 12:04:34.383862 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:34.383867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:34.383936 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:34.384039 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:34.384819 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.385661 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.387097 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.387487 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.387662 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.393375 (+ 5713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:59075: BlockingRecv error: recv got EOF from 127.0.147.66:59075 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:34.404966 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:34.394199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:47809)
0809 12:04:34.395051 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:34.395074 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:34.395266 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:34.395894 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:34.395901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:34.396457 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:34.396730 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.396745 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.398323 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.398328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.399618 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.399631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.400331 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.400337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.400534 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.401263 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:34.401279 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:34.404666 (+ 3387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":4055,"threads_started":1}
W20260809 12:04:34.405125 13457 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:34.394616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:47809 (local address 127.0.147.126:39419)
0809 12:04:34.394826 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:34.394833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:34.395150 (+ 317us) server_negotiation.cc:407] Connection header received
0809 12:04:34.395434 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:34.395442 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:34.395547 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:34.395633 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:34.396936 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.398114 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.399752 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.400175 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.400303 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.404864 (+ 4561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:47809: BlockingRecv error: recv got EOF from 127.0.147.66:47809 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 12:04:34.405215 13453 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:04:34.978981 13640 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.237s user 0.000s sys 0.020s
W20260809 12:04:34.980423 13641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:33.724836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:35585 (local address 127.0.147.126:39419)
0809 12:04:33.727986 (+ 3150us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:33.727994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:33.728917 (+ 923us) server_negotiation.cc:407] Connection header received
0809 12:04:33.732147 (+ 3230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:33.732156 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:33.732255 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:33.732346 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:33.734915 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.736291 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.738895 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.739712 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.739839 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.980163 (+1240324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:35585: BlockingRecv error: recv got EOF from 127.0.147.67:35585 (error 108)
Metrics: {"server-negotiator.queue_time_us":2997,"thread_start_us":2871,"threads_started":1}
I20260809 12:04:34.981160 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:33.724062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:35585)
0809 12:04:33.728640 (+ 4578us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:33.728692 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:33.731790 (+ 3098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:33.733035 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:33.733050 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:33.733909 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:33.734683 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:33.734708 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.736524 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.736530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:33.738724 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:33.738741 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:33.740237 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:33.740248 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:33.740550 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:33.741525 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:33.741563 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:34.979975 (+1238412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4198,"thread_start_us":3921,"threads_started":1}
W20260809 12:04:34.982048 13638 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:39419 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.147.126:39419: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:34.991706 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:34.982689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:60783)
0809 12:04:34.982903 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:34.982925 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:34.983046 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:34.983392 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:34.983397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:34.983851 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:34.984261 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.984277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.985352 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.985356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.986449 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.986462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.987175 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.987179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.987296 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.987937 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:34.987954 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:34.991412 (+ 3458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 12:04:34.991923 13641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:34.982791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:60783 (local address 127.0.147.126:39419)
0809 12:04:34.983000 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:34.983005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:34.983025 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:04:34.983181 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:34.983186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:34.983253 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:34.983358 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:34.984453 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.985178 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.986593 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.987010 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.987118 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.991635 (+ 4517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:60783: BlockingRecv error: recv got EOF from 127.0.147.67:60783 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:34 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:35.000795 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:34.992489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:46465)
0809 12:04:34.992626 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:34.992647 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:34.992753 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:34.993107 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:34.993113 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:34.993461 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:34.993631 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.993639 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.994692 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.994696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.995826 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:34.995839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.996509 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.996514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:34.996637 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:34.997128 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:34.997145 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:35.000593 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 12:04:35.001009 13641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:34.992563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:46465 (local address 127.0.147.126:39419)
0809 12:04:34.992781 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:34.992790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:34.992809 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:04:34.992859 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:34.992864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:34.992960 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:34.993068 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:34.993883 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.994553 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.995949 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:34.996360 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:34.996453 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:35.000764 (+ 4311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:46465: BlockingRecv error: recv got EOF from 127.0.147.67:46465 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
W20260809 12:04:35.001266 13638 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 12:04:35 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:35 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:35.071693 13650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:35.062077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:56063)
0809 12:04:35.062709 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:35.062725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:35.062846 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:35.063257 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:35.063261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:35.063959 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:35.064131 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:35.064140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:35.065472 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:35.065476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:35.066439 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:35.066450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:35.067149 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:35.067153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:35.067280 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:35.067854 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:35.067872 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:35.071362 (+ 3490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":360,"threads_started":1}
W20260809 12:04:35.071841 13641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:35.062252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:56063 (local address 127.0.147.126:39419)
0809 12:04:35.062448 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:35.062452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:35.062843 (+ 391us) server_negotiation.cc:407] Connection header received
0809 12:04:35.062962 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:35.062967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:35.063076 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:35.063197 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:35.064296 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:35.065335 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:35.066567 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:35.066972 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:35.067090 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:35.071623 (+ 4533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:56063: BlockingRecv error: recv got EOF from 127.0.147.65:56063 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 12:04:35 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:35 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:35.415282 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:35.405819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:52199)
0809 12:04:35.406423 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:35.406441 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:35.406603 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:35.406983 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:35.406988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:35.407468 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:35.407699 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:35.407709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:35.408765 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:35.408771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:35.409740 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:35.409751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:35.410548 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:35.410556 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:35.410729 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:35.411377 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:35.411394 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:35.414929 (+ 3535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":336,"threads_started":1}
W20260809 12:04:35.415405 13641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:35.405874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:52199 (local address 127.0.147.126:39419)
0809 12:04:35.406123 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:35.406131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:35.406535 (+ 404us) server_negotiation.cc:407] Connection header received
0809 12:04:35.406737 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:35.406742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:35.406817 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:35.406922 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:35.407843 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:35.408613 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:35.409885 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:35.410329 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:35.410469 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:35.415150 (+ 4681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:52199: BlockingRecv error: recv got EOF from 127.0.147.66:52199 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 12:04:36 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:36 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:36.011384 13652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:36.001789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:38155)
0809 12:04:36.002393 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:36.002410 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:36.002541 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:36.002854 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:36.002857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:36.003291 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:36.003444 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:36.003452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.004545 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.004549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:36.005529 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:36.005540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.006240 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.006244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:36.006359 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:36.006940 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:36.006958 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:36.011063 (+ 4105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":360,"threads_started":1}
W20260809 12:04:36.011637 13654 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:36.001880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:38155 (local address 127.0.147.126:39419)
0809 12:04:36.002362 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:36.002366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:36.002483 (+ 117us) server_negotiation.cc:407] Connection header received
0809 12:04:36.002640 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:36.002643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:36.002716 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:36.002805 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:36.003675 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.004419 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.005654 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.006078 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.006166 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:36.011346 (+ 5180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:38155: BlockingRecv error: recv got EOF from 127.0.147.67:38155 (error 108)
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":278,"threads_started":1}
Aug 09 12:04:36 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:36 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:36.081881 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:36.072702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:51387)
0809 12:04:36.073265 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:36.073281 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:36.073405 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:36.073727 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:36.073730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:36.074157 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:36.074395 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:36.074406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.075454 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.075458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:36.076523 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:36.076535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.077302 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.077306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:36.077432 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:36.078024 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:36.078041 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:36.081498 (+ 3457us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":322,"threads_started":1}
W20260809 12:04:36.082005 13654 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:36.072735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:51387 (local address 127.0.147.126:39419)
0809 12:04:36.072914 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:36.072918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:36.073357 (+ 439us) server_negotiation.cc:407] Connection header received
0809 12:04:36.073504 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:36.073509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:36.073572 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:36.073699 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:36.074522 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.075317 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.076646 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.077150 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.077263 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:36.081706 (+ 4443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:51387: BlockingRecv error: recv got EOF from 127.0.147.65:51387 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 12:04:36 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:36 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:36.425319 13656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:36.416329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:60129)
0809 12:04:36.416830 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:36.416846 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:36.416958 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:36.417228 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:36.417231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:36.417636 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:36.417818 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:36.417827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.418791 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.418795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:36.419732 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:36.419743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.420391 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.420395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:36.420517 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:36.421172 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:36.421189 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:36.425072 (+ 3883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":292,"threads_started":1}
W20260809 12:04:36.425549 13654 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:36.416449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:60129 (local address 127.0.147.126:39419)
0809 12:04:36.416703 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:36.416709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:36.416926 (+ 217us) server_negotiation.cc:407] Connection header received
0809 12:04:36.417072 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:36.417075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:36.417137 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:36.417215 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:36.417942 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.418672 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.419845 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:36.420260 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:36.420352 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:36.425256 (+ 4904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:60129: BlockingRecv error: recv got EOF from 127.0.147.66:60129 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 09 12:04:37 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:37 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:37.021908 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:37.012626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:60105)
0809 12:04:37.013240 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:37.013264 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:37.013415 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:37.013801 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:37.013807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:37.014293 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:37.014482 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:37.014491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.015578 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.015584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:37.016499 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:37.016511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.017214 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.017220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:37.017403 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:37.017995 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:37.018013 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:37.021672 (+ 3659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":339,"threads_started":1}
W20260809 12:04:37.022359 13658 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:37.012767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:60105 (local address 127.0.147.126:39419)
0809 12:04:37.013481 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:37.013489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:37.013506 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:04:37.013570 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:37.013574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:37.013641 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:37.013765 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:37.014644 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.015396 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.016662 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.017057 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.017192 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:37.021932 (+ 4740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:60105: BlockingRecv error: recv got EOF from 127.0.147.67:60105 (error 108)
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":487,"threads_started":1}
Aug 09 12:04:37 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:37 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:37.092193 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:37.082901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:34185)
0809 12:04:37.083382 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:37.083398 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:37.083512 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:37.083925 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:37.083929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:37.084295 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:37.084531 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:37.084540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.085876 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.085880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:37.086866 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:37.086878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.087488 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.087492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:37.087685 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:37.088239 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:37.088256 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:37.091820 (+ 3564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":280,"threads_started":1}
W20260809 12:04:37.092243 13658 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:37.082953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:34185 (local address 127.0.147.126:39419)
0809 12:04:37.083183 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:37.083193 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:37.083480 (+ 287us) server_negotiation.cc:407] Connection header received
0809 12:04:37.083700 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:37.083703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:37.083765 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:37.083879 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:37.084705 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.085758 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.086986 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.087369 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.087460 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:37.092003 (+ 4543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:34185: BlockingRecv error: recv got EOF from 127.0.147.65:34185 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 12:04:37 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:37 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:37.435372 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:37.426381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:44471)
0809 12:04:37.426957 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:37.426996 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:37.427153 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:37.427572 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:37.427577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:37.427980 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:37.428196 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:37.428205 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.429310 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.429313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:37.430297 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:37.430310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.430980 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.430984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:37.431111 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:37.431756 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:37.431773 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:37.435113 (+ 3340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":328,"threads_started":1}
W20260809 12:04:37.435561 13658 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:37.426484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:44471 (local address 127.0.147.126:39419)
0809 12:04:37.426741 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:37.426749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:37.427094 (+ 345us) server_negotiation.cc:407] Connection header received
0809 12:04:37.427264 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:37.427270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:37.427368 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:37.427487 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:37.428324 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.429178 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.430417 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:37.430858 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:37.430951 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:37.435294 (+ 4343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:44471: BlockingRecv error: recv got EOF from 127.0.147.66:44471 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 12:04:38 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:38 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:38.037195 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:38.022892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:48109)
0809 12:04:38.023708 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:38.023726 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:38.023838 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:38.024175 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:38.024178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:38.024546 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:38.024707 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:38.024715 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.026122 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.026125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:38.027028 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:38.027039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.027975 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.027979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:38.028120 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:38.033298 (+ 5178us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:38.033317 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:38.036833 (+ 3516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":515,"threads_started":1}
W20260809 12:04:38.037356 13664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:38.023079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:48109 (local address 127.0.147.126:39419)
0809 12:04:38.023729 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:38.023736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:38.023782 (+ 46us) server_negotiation.cc:407] Connection header received
0809 12:04:38.023948 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:38.023953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:38.024025 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:38.024132 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:38.024850 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.025969 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.027161 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.027848 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.027966 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:38.037052 (+ 9086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:48109: BlockingRecv error: recv got EOF from 127.0.147.67:48109 (error 108)
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":373,"threads_started":1}
Aug 09 12:04:38 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:38 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:38.102479 13666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:38.093307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:46897)
0809 12:04:38.093908 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:38.093924 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:38.094037 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:38.094327 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:38.094331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:38.094742 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:38.094917 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:38.094925 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.095906 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.095910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:38.096908 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:38.096920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.097554 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.097559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:38.097681 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:38.098273 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:38.098290 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:38.102108 (+ 3818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":356,"threads_started":1}
W20260809 12:04:38.102528 13664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:38.093364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:46897 (local address 127.0.147.126:39419)
0809 12:04:38.093574 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:38.093578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:38.093995 (+ 417us) server_negotiation.cc:407] Connection header received
0809 12:04:38.094134 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:38.094138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:38.094200 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:38.094282 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:38.095037 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.095781 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.097031 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.097417 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.097503 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:38.102308 (+ 4805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:46897: BlockingRecv error: recv got EOF from 127.0.147.65:46897 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 09 12:04:38 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:38 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:38.446085 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:38.436527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:59545)
0809 12:04:38.437090 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:38.437107 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:38.437258 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:38.437618 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:38.437622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:38.438048 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:38.438225 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:38.438235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.439287 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.439295 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:38.440329 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:38.440341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.441067 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.441074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:38.441264 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:38.441863 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:38.441881 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:38.445719 (+ 3838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":317,"threads_started":1}
W20260809 12:04:38.446183 13664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:38.436612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:59545 (local address 127.0.147.126:39419)
0809 12:04:38.436840 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:38.436848 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:38.437194 (+ 346us) server_negotiation.cc:407] Connection header received
0809 12:04:38.437380 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:38.437389 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:38.437466 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:38.437572 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:38.438368 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.439157 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.440475 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:38.440895 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:38.441048 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:38.445921 (+ 4873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:59545: BlockingRecv error: recv got EOF from 127.0.147.66:59545 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 12:04:39 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:39 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:39.046849 13670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:39.038149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:36037)
0809 12:04:39.038717 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:39.038735 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:39.038836 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:39.039174 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:39.039178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:39.039636 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:39.039805 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:39.039814 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.040837 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.040841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:39.041860 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:39.041872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.042516 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.042520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:39.042642 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:39.043216 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:39.043234 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:39.046603 (+ 3369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":288,"threads_started":1}
W20260809 12:04:39.047175 13672 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:39.038223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:36037 (local address 127.0.147.126:39419)
0809 12:04:39.038758 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:39.038762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:39.038781 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:04:39.038941 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:39.038945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:39.039004 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:39.039117 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:39.039938 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.040711 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.041993 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.042396 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.042489 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:39.046859 (+ 4370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:36037: BlockingRecv error: recv got EOF from 127.0.147.67:36037 (error 108)
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":307,"threads_started":1}
Aug 09 12:04:39 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:39 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:39.112912 13674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:39.103512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:47441)
0809 12:04:39.104141 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:39.104157 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:39.104311 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:39.104648 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:39.104655 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:39.105113 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:39.105278 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:39.105286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.106351 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.106357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:39.107436 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:39.107448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.108345 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.108366 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:39.108585 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:39.109274 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:39.109292 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:39.112572 (+ 3280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":368,"threads_started":1}
W20260809 12:04:39.113031 13672 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:39.103646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:47441 (local address 127.0.147.126:39419)
0809 12:04:39.103972 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:39.103978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:39.104237 (+ 259us) server_negotiation.cc:407] Connection header received
0809 12:04:39.104429 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:39.104434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:39.104505 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:39.104609 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:39.105420 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.106214 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.107639 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.108203 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.108335 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:39.112757 (+ 4422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:47441: BlockingRecv error: recv got EOF from 127.0.147.65:47441 (error 108)
Metrics: {"server-negotiator.queue_time_us":169}
Aug 09 12:04:39 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:39 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:39.456557 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:39.447124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:52269)
0809 12:04:39.447716 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:39.447733 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:39.447892 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:39.448267 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:39.448272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:39.448732 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:39.448888 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:39.448898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.450337 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.450346 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:39.451357 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:39.451370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.452128 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.452134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:39.452315 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:39.452863 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:39.452880 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:39.456325 (+ 3445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":264,"threads_started":1}
W20260809 12:04:39.456806 13672 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:39.447389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:52269 (local address 127.0.147.126:39419)
0809 12:04:39.447661 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:39.447668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:39.447815 (+ 147us) server_negotiation.cc:407] Connection header received
0809 12:04:39.448041 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:39.448045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:39.448122 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:39.448229 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:39.449061 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.450192 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.451503 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:39.451973 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:39.452108 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:39.456517 (+ 4409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:52269: BlockingRecv error: recv got EOF from 127.0.147.66:52269 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 12:04:40 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:40 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:40.057371 13676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:40.047956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:45385)
0809 12:04:40.048592 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:40.048614 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:40.048744 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:40.049109 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:40.049112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:40.049524 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:40.049671 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:40.049679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.050691 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.050694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:40.051885 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:40.051902 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.052903 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.052909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:40.053054 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:40.053647 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:40.053665 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:40.057140 (+ 3475us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":294,"threads_started":1}
W20260809 12:04:40.057683 13678 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:40.048041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:45385 (local address 127.0.147.126:39419)
0809 12:04:40.048791 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:40.048797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:40.048818 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:04:40.048874 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:40.048879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:40.048953 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:40.049073 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:40.049814 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.050564 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.052092 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.052738 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.052874 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:40.057410 (+ 4536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:45385: BlockingRecv error: recv got EOF from 127.0.147.67:45385 (error 108)
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":526,"threads_started":1}
Aug 09 12:04:40 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:40 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:40.125185 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:40.114039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:34767)
0809 12:04:40.114567 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:40.114586 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:40.114714 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:40.115074 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:40.115077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:40.115514 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:40.115704 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:40.115719 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.117151 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.117157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:40.118879 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:40.118899 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.119986 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.119993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:40.120158 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:40.120743 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:40.120762 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:40.124830 (+ 4068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":310,"threads_started":1}
W20260809 12:04:40.125273 13678 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:40.114206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:34767 (local address 127.0.147.126:39419)
0809 12:04:40.114491 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:40.114498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:40.114673 (+ 175us) server_negotiation.cc:407] Connection header received
0809 12:04:40.114816 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:40.114820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:40.114885 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:40.115015 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:40.115862 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.116972 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.119089 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.119782 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.119940 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:40.125039 (+ 5099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:34767: BlockingRecv error: recv got EOF from 127.0.147.65:34767 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 09 12:04:40 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:40 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:40.466703 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:40.457571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:46883)
0809 12:04:40.458186 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:40.458209 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:40.458369 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:40.458733 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:40.458737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:40.459183 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:40.459360 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:40.459370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.460501 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.460507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:40.461476 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:40.461487 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.462214 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.462220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:40.462396 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:40.463016 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:40.463033 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:40.466360 (+ 3327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":378,"threads_started":1}
W20260809 12:04:40.466822 13678 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:40.457573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:46883 (local address 127.0.147.126:39419)
0809 12:04:40.457784 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:40.457794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:40.458299 (+ 505us) server_negotiation.cc:407] Connection header received
0809 12:04:40.458494 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:40.458499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:40.458580 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:40.458686 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:40.459510 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.460357 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.461629 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:40.462042 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:40.462177 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:40.466555 (+ 4378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:46883: BlockingRecv error: recv got EOF from 127.0.147.66:46883 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 12:04:41 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:41 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:41.067826 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:41.058343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:44571)
0809 12:04:41.058895 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:41.058914 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:41.059021 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:41.059333 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:41.059337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:41.059852 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:41.060045 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:41.060056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.061133 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.061137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:41.062106 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:41.062118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.062758 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.062799 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:41.062967 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:41.063643 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:41.063681 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:41.067464 (+ 3783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":306,"threads_started":1}
W20260809 12:04:41.067993 13682 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:41.058409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:44571 (local address 127.0.147.126:39419)
0809 12:04:41.058938 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:41.058943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:41.058963 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:04:41.059135 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:41.059139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:41.059199 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:41.059287 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:41.060191 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.060988 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.062225 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.062625 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.062716 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:41.067695 (+ 4979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:44571: BlockingRecv error: recv got EOF from 127.0.147.67:44571 (error 108)
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":318,"threads_started":1}
Aug 09 12:04:41 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:41 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:41.135075 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:41.126146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:56489)
0809 12:04:41.126829 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:41.126845 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:41.126961 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:41.127241 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:41.127244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:41.127695 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:41.127883 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:41.127892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.128921 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.128926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:41.130036 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:41.130050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.130690 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.130694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:41.130808 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:41.131352 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:41.131369 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:41.134760 (+ 3391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":317,"threads_started":1}
W20260809 12:04:41.135124 13682 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:41.126217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:56489 (local address 127.0.147.126:39419)
0809 12:04:41.126387 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:41.126391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:41.126913 (+ 522us) server_negotiation.cc:407] Connection header received
0809 12:04:41.127055 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:41.127058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:41.127118 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:41.127199 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:41.128021 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.128779 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.130159 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.130558 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.130654 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:41.134934 (+ 4280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:56489: BlockingRecv error: recv got EOF from 127.0.147.65:56489 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 12:04:41 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:41 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:41.476679 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:41.467748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:48345)
0809 12:04:41.468410 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:41.468426 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:41.468543 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:41.468904 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:41.468908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:41.469363 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:41.469535 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:41.469544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.470612 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.470618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:41.471571 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:41.471582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.472303 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.472309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:41.472483 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:41.473019 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:41.473035 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:41.476444 (+ 3409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":326,"threads_started":1}
W20260809 12:04:41.477046 13682 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:41.467807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:48345 (local address 127.0.147.126:39419)
0809 12:04:41.468049 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:41.468055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:41.468509 (+ 454us) server_negotiation.cc:407] Connection header received
0809 12:04:41.468690 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:41.468695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:41.468765 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:41.468865 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:41.469675 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.470471 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.471708 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:41.472145 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:41.472272 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:41.476629 (+ 4357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:48345: BlockingRecv error: recv got EOF from 127.0.147.66:48345 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 09 12:04:42 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:42 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:42.078810 13686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:42.068822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:52505)
0809 12:04:42.069442 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:42.069459 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:42.069583 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:42.069903 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:42.069906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:42.070347 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:42.070564 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:42.070579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.072158 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.072162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:42.073120 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:42.073132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.073848 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.073853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:42.073973 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:42.074609 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:42.074627 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:42.078391 (+ 3764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":309,"threads_started":1}
W20260809 12:04:42.078884 13691 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:42.068935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:52505 (local address 127.0.147.126:39419)
0809 12:04:42.069463 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:42.069468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:42.069541 (+ 73us) server_negotiation.cc:407] Connection header received
0809 12:04:42.069686 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:42.069690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:42.069762 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:42.069855 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:42.070737 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.071996 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.073263 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.073705 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.073803 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:42.078628 (+ 4825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:52505: BlockingRecv error: recv got EOF from 127.0.147.67:52505 (error 108)
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":305,"threads_started":1}
Aug 09 12:04:42 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:42 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:42.145195 13692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:42.136021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:53459)
0809 12:04:42.136549 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:42.136567 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:42.136688 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:42.136973 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:42.136976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:42.137403 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:42.137577 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:42.137585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.138643 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.138647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:42.139642 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:42.139654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.140317 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.140322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:42.140435 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:42.141078 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:42.141095 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:42.144940 (+ 3845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":266,"threads_started":1}
W20260809 12:04:42.145365 13691 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:42.136162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:53459 (local address 127.0.147.126:39419)
0809 12:04:42.136386 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:42.136391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:42.136641 (+ 250us) server_negotiation.cc:407] Connection header received
0809 12:04:42.136783 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:42.136787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:42.136850 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:42.136930 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:42.137694 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.138523 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.139756 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.140169 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.140261 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:42.145127 (+ 4866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:53459: BlockingRecv error: recv got EOF from 127.0.147.65:53459 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 12:04:42 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:42 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:42.487172 13693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:42.477727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:37679)
0809 12:04:42.478318 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:42.478335 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:42.478452 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:42.478757 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:42.478761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:42.479189 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:42.479356 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:42.479364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.480551 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.480556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:42.481543 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:42.481555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.482268 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.482272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:42.482382 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:42.482990 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:42.483008 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:42.486922 (+ 3914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":349,"threads_started":1}
W20260809 12:04:42.487385 13691 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:42.477806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:37679 (local address 127.0.147.126:39419)
0809 12:04:42.478028 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:42.478037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:42.478424 (+ 387us) server_negotiation.cc:407] Connection header received
0809 12:04:42.478551 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:42.478555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:42.478632 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:42.478717 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:42.479491 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.480386 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.481677 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:42.482116 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:42.482215 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:42.487152 (+ 4937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:37679: BlockingRecv error: recv got EOF from 127.0.147.66:37679 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 12:04:43 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:43 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:43.093528 13695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:43.079834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:48155)
0809 12:04:43.080361 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:43.080379 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:43.080481 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:43.080962 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:43.080966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:43.081382 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:43.081562 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:43.081570 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.082594 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.082598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:43.083519 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:43.083554 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.084222 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.084226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:43.084334 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:43.089637 (+ 5303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:43.089656 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:43.093288 (+ 3632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":275,"threads_started":1}
W20260809 12:04:43.093771 13697 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:43.079903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:48155 (local address 127.0.147.126:39419)
0809 12:04:43.080694 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:43.080698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:43.080713 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:04:43.080756 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:43.080759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:43.080814 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:43.080913 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:43.081695 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.082481 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.083659 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.084075 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.084165 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:43.093471 (+ 9306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:48155: BlockingRecv error: recv got EOF from 127.0.147.67:48155 (error 108)
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":589,"threads_started":1}
Aug 09 12:04:43 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:43 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:43.155104 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:43.146244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:49613)
0809 12:04:43.146718 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:43.146734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:43.146845 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:43.147156 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:43.147162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:43.147683 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:43.147838 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:43.147847 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.148902 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.148912 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:43.149859 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:43.149871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.150570 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.150576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:43.150745 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:43.151313 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:43.151330 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:43.154778 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":234,"threads_started":1}
W20260809 12:04:43.155189 13697 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:43.146306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:49613 (local address 127.0.147.126:39419)
0809 12:04:43.146536 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:43.146542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:43.146811 (+ 269us) server_negotiation.cc:407] Connection header received
0809 12:04:43.146937 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:43.146940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:43.147001 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:43.147126 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:43.147981 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.148762 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.149997 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.150414 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.150544 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:43.154960 (+ 4416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:49613: BlockingRecv error: recv got EOF from 127.0.147.65:49613 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 12:04:43 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:43 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:43.497422 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:43.488181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:39395)
0809 12:04:43.488742 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:43.488758 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:43.488911 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:43.489347 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:43.489352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:43.489838 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:43.490016 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:43.490025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.490991 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.490994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:43.492028 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:43.492040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.492815 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.492819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:43.492937 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:43.493474 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:43.493490 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:43.497100 (+ 3610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":313,"threads_started":1}
W20260809 12:04:43.497566 13697 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:43.488255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:39395 (local address 127.0.147.126:39419)
0809 12:04:43.488468 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:43.488474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:43.488842 (+ 368us) server_negotiation.cc:407] Connection header received
0809 12:04:43.489086 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:43.489091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:43.489176 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:43.489286 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:43.490146 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.490880 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.492149 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:43.492698 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:43.492788 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:43.497266 (+ 4478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:39395: BlockingRecv error: recv got EOF from 127.0.147.66:39395 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 12:04:44 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:44 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:44.103078 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:44.094396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:37805)
0809 12:04:44.094964 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:44.094982 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:44.095093 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:44.095399 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:44.095402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:44.095839 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:44.095987 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:44.095996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.096981 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.096985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:44.097939 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:44.097950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.098611 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.098615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:44.098740 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:44.099300 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:44.099318 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:44.102845 (+ 3527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":265,"threads_started":1}
W20260809 12:04:44.103300 13705 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:44.094490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:37805 (local address 127.0.147.126:39419)
0809 12:04:44.095013 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:44.095020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:44.095039 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:04:44.095202 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:44.095206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:44.095265 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:44.095353 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:44.096117 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.096852 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.098064 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.098475 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.098575 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:44.103056 (+ 4481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:37805: BlockingRecv error: recv got EOF from 127.0.147.67:37805 (error 108)
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":308,"threads_started":1}
Aug 09 12:04:44 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:44 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:44.165423 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:44.156181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:57069)
0809 12:04:44.156730 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:44.156747 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:44.156862 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:44.157166 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:44.157169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:44.157562 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:44.157716 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:44.157724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.158990 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.158995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:44.160317 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:44.160332 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.161062 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.161068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:44.161237 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:44.161900 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:44.161919 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:44.165182 (+ 3263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":292,"threads_started":1}
W20260809 12:04:44.165581 13705 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:44.156275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:57069 (local address 127.0.147.126:39419)
0809 12:04:44.156471 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:44.156475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:44.156817 (+ 342us) server_negotiation.cc:407] Connection header received
0809 12:04:44.156957 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:44.156960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:44.157040 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:44.157125 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:44.157842 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.158819 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.160465 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.160884 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.161064 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:44.165352 (+ 4288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:57069: BlockingRecv error: recv got EOF from 127.0.147.65:57069 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 12:04:44 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:44 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:44.507825 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:44.498355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:45689)
0809 12:04:44.498977 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:44.498994 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:44.499219 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:44.499636 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:44.499641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:44.500069 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:44.500244 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:44.500253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.501415 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.501421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:44.502534 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:44.502547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.503278 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.503284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:44.503471 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:44.504137 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:44.504155 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:44.507516 (+ 3361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":302,"threads_started":1}
W20260809 12:04:44.507987 13705 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:44.498466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:45689 (local address 127.0.147.126:39419)
0809 12:04:44.498737 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:44.498743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:44.499125 (+ 382us) server_negotiation.cc:407] Connection header received
0809 12:04:44.499359 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:44.499365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:44.499468 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:44.499610 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:44.500400 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.501263 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.502675 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:44.503116 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:44.503247 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:44.507736 (+ 4489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:45689: BlockingRecv error: recv got EOF from 127.0.147.66:45689 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 09 12:04:45 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:45 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:45.112947 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:45.103997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:38817)
0809 12:04:45.104547 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:45.104565 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:45.104669 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:45.105145 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:45.105149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:45.105547 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:45.105714 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:45.105724 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.106753 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.106757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:45.107785 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:45.107798 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.108451 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.108455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:45.108571 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:45.109154 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:45.109172 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:45.112579 (+ 3407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":314,"threads_started":1}
W20260809 12:04:45.113354 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:45.104273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:38817 (local address 127.0.147.126:39419)
0809 12:04:45.104884 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:45.104889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:45.104904 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:04:45.104946 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:45.104950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:45.105004 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:45.105094 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:45.105866 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.106616 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.107908 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.108315 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.108406 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:45.112841 (+ 4435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:38817: BlockingRecv error: recv got EOF from 127.0.147.67:38817 (error 108)
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":398,"threads_started":1}
Aug 09 12:04:45 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:45 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:45.176230 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:45.166418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:59161)
0809 12:04:45.167028 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:45.167047 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:45.167175 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:45.167484 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:45.167487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:45.168045 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:45.168242 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:45.168252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.169221 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.169226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:45.170258 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:45.170271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.171009 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.171014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:45.171159 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:45.171837 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:45.171874 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:45.175993 (+ 4119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":318,"threads_started":1}
W20260809 12:04:45.176350 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:45.166476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:59161 (local address 127.0.147.126:39419)
0809 12:04:45.166653 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:45.166657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:45.167126 (+ 469us) server_negotiation.cc:407] Connection header received
0809 12:04:45.167280 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:45.167284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:45.167345 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:45.167426 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:45.168388 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.169103 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.170392 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.170858 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.170951 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:45.176156 (+ 5205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:59161: BlockingRecv error: recv got EOF from 127.0.147.65:59161 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 12:04:45 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:45 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:45.518117 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:45.508935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:38005)
0809 12:04:45.509467 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:45.509484 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:45.509598 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:45.509912 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:45.509915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:45.510356 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:45.510542 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:45.510550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.511519 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.511578 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:45.512479 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:45.512491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.513149 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.513152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:45.513281 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:45.513917 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:45.513933 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:45.517780 (+ 3847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":308,"threads_started":1}
W20260809 12:04:45.518254 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:45.509052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:38005 (local address 127.0.147.126:39419)
0809 12:04:45.509222 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:45.509226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:45.509555 (+ 329us) server_negotiation.cc:407] Connection header received
0809 12:04:45.509692 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:45.509696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:45.509773 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:45.509866 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:45.510665 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.511406 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.512594 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:45.513020 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:45.513110 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:45.517981 (+ 4871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:38005: BlockingRecv error: recv got EOF from 127.0.147.66:38005 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 12:04:46 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:46 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:46.123585 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:46.113922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:43923)
0809 12:04:46.114679 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:46.114696 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:46.114809 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:46.115103 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:46.115106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:46.115589 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:46.115786 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:46.115796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.116870 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.116873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:46.117876 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:46.117890 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.118637 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.118641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:46.118762 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:46.119358 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:46.119376 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:46.123172 (+ 3796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":308,"threads_started":1}
W20260809 12:04:46.123705 13716 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:46.113992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:43923 (local address 127.0.147.126:39419)
0809 12:04:46.114672 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:46.114677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:46.114752 (+ 75us) server_negotiation.cc:407] Connection header received
0809 12:04:46.114904 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:46.114907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:46.114969 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:46.115063 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:46.115914 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.116744 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.118014 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.118486 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.118611 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:46.123388 (+ 4777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:43923: BlockingRecv error: recv got EOF from 127.0.147.67:43923 (error 108)
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":426,"threads_started":1}
Aug 09 12:04:46 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:46 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:46.186718 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:46.177179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:52595)
0809 12:04:46.177737 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:46.177754 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:46.177871 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:46.178173 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:46.178176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:46.178582 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:46.178769 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:46.178777 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.179793 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.179798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:46.180720 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:46.180731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.181390 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.181396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:46.181593 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:46.182202 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:46.182219 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:46.186328 (+ 4109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":323,"threads_started":1}
W20260809 12:04:46.186782 13716 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:46.177225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:52595 (local address 127.0.147.126:39419)
0809 12:04:46.177386 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:46.177391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:46.177826 (+ 435us) server_negotiation.cc:407] Connection header received
0809 12:04:46.177971 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:46.177975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:46.178038 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:46.178122 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:46.178888 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.179670 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.180838 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.181234 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.181380 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:46.186531 (+ 5151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:52595: BlockingRecv error: recv got EOF from 127.0.147.65:52595 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 12:04:46 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:46 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:46.528406 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:46.519243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:52051)
0809 12:04:46.519792 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:46.519810 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:46.519925 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:46.520224 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:46.520228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:46.520634 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:46.520810 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:46.520819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.521814 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.521817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:46.522741 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:46.522753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.523483 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.523490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:46.523696 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:46.524273 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:46.524289 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:46.528179 (+ 3890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":325,"threads_started":1}
W20260809 12:04:46.528632 13716 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:46.519293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:52051 (local address 127.0.147.126:39419)
0809 12:04:46.519542 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:46.519552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:46.519887 (+ 335us) server_negotiation.cc:407] Connection header received
0809 12:04:46.520019 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:46.520023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:46.520102 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:46.520181 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:46.520945 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.521696 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.522859 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:46.523289 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:46.523425 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:46.528382 (+ 4957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:52051: BlockingRecv error: recv got EOF from 127.0.147.66:52051 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 12:04:47 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:47 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:47.134709 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:47.124682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:37605)
0809 12:04:47.125244 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:47.125263 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:47.125370 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:47.126046 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:47.126050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:47.126539 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:47.126770 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:47.126783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.127987 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.127991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:47.129004 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:47.129017 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.129956 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.129960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:47.130096 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:47.130680 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:47.130697 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:47.134428 (+ 3731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":302,"threads_started":1}
W20260809 12:04:47.134892 13723 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:47.124713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:37605 (local address 127.0.147.126:39419)
0809 12:04:47.125694 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:47.125700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:47.125720 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:04:47.125775 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:47.125782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:47.125871 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:47.125965 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:47.126929 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.127859 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.129151 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.129793 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.129906 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:47.134649 (+ 4743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:37605: BlockingRecv error: recv got EOF from 127.0.147.67:37605 (error 108)
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":742,"threads_started":1}
Aug 09 12:04:47 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:47 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:47.196271 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:47.187777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:54671)
0809 12:04:47.188248 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:47.188264 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:47.188378 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:47.188666 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:47.188669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:47.189074 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:47.189225 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:47.189234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.190193 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.190197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:47.191115 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:47.191126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.191836 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.191840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:47.192029 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:47.192596 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:47.192613 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:47.195972 (+ 3359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":266,"threads_started":1}
W20260809 12:04:47.196353 13723 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:47.187853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:54671 (local address 127.0.147.126:39419)
0809 12:04:47.188096 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:47.188102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:47.188339 (+ 237us) server_negotiation.cc:407] Connection header received
0809 12:04:47.188467 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:47.188470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:47.188531 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:47.188616 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:47.189343 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.190055 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.191234 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.191719 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.191811 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:47.196159 (+ 4348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:54671: BlockingRecv error: recv got EOF from 127.0.147.65:54671 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 12:04:47 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:47 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:47.538195 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:47.529370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:44197)
0809 12:04:47.529871 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:47.529888 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:47.530001 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:47.530364 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:47.530367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:47.530801 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:47.530964 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:47.530973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.532092 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.532095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:47.533004 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:47.533015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.533720 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.533724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:47.533846 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:47.534430 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:47.534447 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:47.537939 (+ 3492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":281,"threads_started":1}
W20260809 12:04:47.538522 13723 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:47.529523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:44197 (local address 127.0.147.126:39419)
0809 12:04:47.529703 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:47.529707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:47.529961 (+ 254us) server_negotiation.cc:407] Connection header received
0809 12:04:47.530147 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:47.530151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:47.530230 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:47.530317 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:47.531125 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.531969 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.533149 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:47.533593 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:47.533691 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:47.538202 (+ 4511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:44197: BlockingRecv error: recv got EOF from 127.0.147.66:44197 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 12:04:48 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:48 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.67@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:48.151850 13726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:48.135736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.67:49283)
0809 12:04:48.136210 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:48.136227 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:48.136330 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:48.136840 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:48.136844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:48.137269 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:48.137436 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:48.137448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.138510 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.138516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:48.139484 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:48.139496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.140272 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.140278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:48.140474 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:48.147806 (+ 7332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:48.147825 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:48.151623 (+ 3798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":238,"threads_started":1}
W20260809 12:04:48.152084 13730 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:48.135772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:49283 (local address 127.0.147.126:39419)
0809 12:04:48.136553 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:48.136557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:48.136590 (+ 33us) server_negotiation.cc:407] Connection header received
0809 12:04:48.136643 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:48.136647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:48.136703 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:48.136790 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:48.137608 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.138368 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.139693 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.140119 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.140249 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:48.151838 (+ 11589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.67:49283: BlockingRecv error: recv got EOF from 127.0.147.67:49283 (error 108)
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":559,"threads_started":1}
Aug 09 12:04:48 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:48 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.65@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:48.206869 13732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:48.197627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.65:55719)
0809 12:04:48.198163 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:48.198179 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:48.198329 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:48.198673 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:48.198678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:48.199091 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:48.199276 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:48.199285 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.200397 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.200403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:48.201361 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:48.201372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.202070 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.202077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:48.202253 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:48.202904 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:48.202922 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:48.206612 (+ 3690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":273,"threads_started":1}
W20260809 12:04:48.207031 13730 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:48.197694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:55719 (local address 127.0.147.126:39419)
0809 12:04:48.198003 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:48.198009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:48.198261 (+ 252us) server_negotiation.cc:407] Connection header received
0809 12:04:48.198455 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:48.198467 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:48.198528 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:48.198635 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:48.199420 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.200251 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.201511 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.201920 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.202048 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:48.206805 (+ 4757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.65:55719: BlockingRecv error: recv got EOF from 127.0.147.65:55719 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
Aug 09 12:04:48 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:04:48 dist-test-slave-sb2z krb5kdc[12270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.147.66@KRBTEST.COM for oryx/127.0.147.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:04:48.549043 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:48.539610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39419 (local address 127.0.147.66:37419)
0809 12:04:48.540184 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:48.540201 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:48.540312 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:48.540650 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:48.540653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:48.541063 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:48.541248 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:48.541257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.542676 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.542680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:48.543675 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:48.543687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.544383 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.544387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:48.544514 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:48.545121 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:48.545137 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:48.548750 (+ 3613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.147.126:39419: Server oryx/127.0.147.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":311,"threads_started":1}
W20260809 12:04:48.549327 13730 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:48.539733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:37419 (local address 127.0.147.126:39419)
0809 12:04:48.539962 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:48.539969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:48.540289 (+ 320us) server_negotiation.cc:407] Connection header received
0809 12:04:48.540400 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:48.540405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:48.540495 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:48.540606 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:48.541399 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.542527 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.543824 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.544258 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.544359 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:48.549002 (+ 4643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.147.66:37419: BlockingRecv error: recv got EOF from 127.0.147.66:37419 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:04:48.736048 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 13078
I20260809 12:04:48.748709 13737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:48.737208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.0.147.65:39915)
0809 12:04:48.737818 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:48.737825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:48.737850 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:04:48.737909 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:48.737915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:48.738164 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:48.738343 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:48.739810 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.740830 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.741993 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.742528 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.742648 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:48.745221 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:48.745229 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:48.745242 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:48.745288 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:48.748163 (+ 2875us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.147.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 12:04:48.748457 (+ 294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54594: Request ticket server kudu/127.0.147.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":341,"threads_started":1}
W20260809 12:04:48.748831 13737 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54594: Request ticket server kudu/127.0.147.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 12:04:48.748852 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 13078: Not authorized: Client connection negotiation failed: client connection to 127.0.147.65:39915: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.147.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:04:48.749096 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 13078
I20260809 12:04:48.777594 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 13288
I20260809 12:04:48.790720 13739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:48.778819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.0.147.66:37479)
0809 12:04:48.779651 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:48.779661 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:48.779700 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:04:48.779781 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:48.779793 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:48.780131 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:48.780361 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:48.781713 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.782881 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.784215 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.784800 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.784910 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:48.787443 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:48.787452 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:48.787465 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:48.787508 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:48.790149 (+ 2641us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.147.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 12:04:48.790434 (+ 285us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50786: Request ticket server kudu/127.0.147.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":364,"threads_started":1}
W20260809 12:04:48.790832 13739 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50786: Request ticket server kudu/127.0.147.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 12:04:48.790827 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 13288: Not authorized: Client connection negotiation failed: client connection to 127.0.147.66:37479: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.147.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:04:48.790987 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 13288
I20260809 12:04:48.818405 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 13462
W20260809 12:04:48.831634 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 13462: Not authorized: Client connection negotiation failed: client connection to 127.0.147.67:42997: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.147.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:04:48.831772 13740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:48.819660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.0.147.67:42997)
0809 12:04:48.820248 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:48.820256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:48.820282 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:04:48.820357 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:48.820363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:48.820611 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:48.820841 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:48.822296 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.823470 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.824722 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:48.825286 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:48.825436 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:48.828164 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:48.828173 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:48.828186 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:48.828220 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:48.830817 (+ 2597us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.147.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 12:04:48.831224 (+ 407us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58462: Request ticket server kudu/127.0.147.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":279,"threads_started":1}
I20260809 12:04:48.831842 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 13462
W20260809 12:04:48.831864 13740 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58462: Request ticket server kudu/127.0.147.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:04:48.857610 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 12994
I20260809 12:04:48.859002 13065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:48.979703 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 12994
2026-08-09T12:04:49Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21863 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T12:04:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:04:49Z Disabled control of system clock
I20260809 12:04:49.080125 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:41477
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:37111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:41477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 12:04:49.416787 13745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:49.417065 13745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:49.417204 13745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:49.424358 13745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:49.424481 13745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:49.424549 13745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:49.424652 13745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:49.424734 13745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:49.433724 13745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:41477
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:41477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:49.435027 13745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:49.436785 13745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:49.448585 13759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:49.448683 13753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:49.448683 13754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:49.449607 13745 server_base.cc:1034] running on GCE node
I20260809 12:04:49.450356 13745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:49.451735 13745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:49.452956 13745 hybrid_clock.cc:648] HybridClock initialized: now 1786277089452897 us; error 49 us; skew 500 ppm
I20260809 12:04:49.453328 13745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:04:49.455672 13745 webserver.cc:460] Webserver started at http://127.0.147.126:35925/ using document root <none> and password file <none>
I20260809 12:04:49.456383 13745 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:49.456509 13745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:49.456862 13745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:49.459905 13745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "d8df123db90e4c3d9898cb9d0a9b8b41"
format_stamp: "Formatted at 2026-08-09 12:04:49 on dist-test-slave-sb2z"
server_key: "740398208b412fbfa5380015deaca00f"
server_key_iv: "9e6e6d6aeb1048c56c3897ccf326d4a3"
server_key_version: "encryptionkey@0"
I20260809 12:04:49.460701 13745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "d8df123db90e4c3d9898cb9d0a9b8b41"
format_stamp: "Formatted at 2026-08-09 12:04:49 on dist-test-slave-sb2z"
server_key: "740398208b412fbfa5380015deaca00f"
server_key_iv: "9e6e6d6aeb1048c56c3897ccf326d4a3"
server_key_version: "encryptionkey@0"
I20260809 12:04:49.465852 13745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:04:49.469225 13765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:49.470454 13745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:04:49.470642 13745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "d8df123db90e4c3d9898cb9d0a9b8b41"
format_stamp: "Formatted at 2026-08-09 12:04:49 on dist-test-slave-sb2z"
server_key: "740398208b412fbfa5380015deaca00f"
server_key_iv: "9e6e6d6aeb1048c56c3897ccf326d4a3"
server_key_version: "encryptionkey@0"
I20260809 12:04:49.470865 13745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:49.488469 13745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:49.489298 13745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:49.489610 13745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:49.508176 13745 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:41477
I20260809 12:04:49.508213 13837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:41477 every 8 connection(s)
I20260809 12:04:49.509476 13745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:04:49.513044 13838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:49.516798 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 13745
I20260809 12:04:49.517004 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:04:49.517863 589 external_mini_cluster.cc:1428] Setting key 5e29b20aa16b05958f122a3ff4868a25
I20260809 12:04:49.526710 13838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41: Bootstrap starting.
I20260809 12:04:49.530426 13838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:49.531629 13838 log.cc:824] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:49.532104 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:49.520038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56382 (local address 127.0.147.126:41477)
0809 12:04:49.521069 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:49.521079 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:49.521121 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:04:49.522665 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:49.522688 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:49.523152 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:49.523485 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:49.524236 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:49.525555 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:49.527434 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:49.528144 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:49.528296 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:49.529648 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:49.529657 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:49.529673 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:49.530215 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:49.530315 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:49.530342 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:49.530432 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:49.530732 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 12:04:49.531066 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":426,"threads_started":1}
I20260809 12:04:49.534994 13838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41: No bootstrap required, opened a new log
I20260809 12:04:49.542675 13838 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df123db90e4c3d9898cb9d0a9b8b41" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 41477 } }
I20260809 12:04:49.543018 13838 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:49.543093 13838 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8df123db90e4c3d9898cb9d0a9b8b41, State: Initialized, Role: FOLLOWER
I20260809 12:04:49.543625 13838 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df123db90e4c3d9898cb9d0a9b8b41" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 41477 } }
I20260809 12:04:49.543830 13838 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:04:49.543926 13838 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:04:49.544052 13838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:49.546298 13838 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df123db90e4c3d9898cb9d0a9b8b41" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 41477 } }
I20260809 12:04:49.546892 13838 leader_election.cc:302] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8df123db90e4c3d9898cb9d0a9b8b41; no voters:
I20260809 12:04:49.547600 13838 leader_election.cc:288] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:04:49.547783 13844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:49.548496 13844 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [term 1 LEADER]: Becoming Leader. State: Replica: d8df123db90e4c3d9898cb9d0a9b8b41, State: Running, Role: LEADER
I20260809 12:04:49.549000 13844 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df123db90e4c3d9898cb9d0a9b8b41" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 41477 } }
I20260809 12:04:49.549738 13838 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:49.552029 13848 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8df123db90e4c3d9898cb9d0a9b8b41. Latest consensus state: current_term: 1 leader_uuid: "d8df123db90e4c3d9898cb9d0a9b8b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df123db90e4c3d9898cb9d0a9b8b41" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 41477 } } }
I20260809 12:04:49.552495 13848 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:49.552341 13846 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8df123db90e4c3d9898cb9d0a9b8b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df123db90e4c3d9898cb9d0a9b8b41" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 41477 } } }
I20260809 12:04:49.552695 13846 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:49.555208 13856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:49.560441 13856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:49.569113 13856 catalog_manager.cc:1347] Generated new cluster ID: ea8a848ef069468c9b40a21ebe581c68
I20260809 12:04:49.569226 13856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:49.596560 13856 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:04:49.597863 13856 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:49.606658 13856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8df123db90e4c3d9898cb9d0a9b8b41: Generated new TSK 0
I20260809 12:04:49.607707 13856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:04:49.615986 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:41477
--builtin_ntp_servers=127.0.147.84:37111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 12:04:49.959271 13869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:49.959628 13869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:49.959789 13869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:49.966989 13869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:49.967108 13869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:49.967295 13869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:04:49.976529 13869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:41477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:49.977739 13869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:49.979437 13869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:49.992337 13878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:49.992522 13876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:49.992465 13880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:49.993502 13869 server_base.cc:1034] running on GCE node
I20260809 12:04:49.994064 13869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:49.995019 13869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:49.996243 13869 hybrid_clock.cc:648] HybridClock initialized: now 1786277089996200 us; error 57 us; skew 500 ppm
I20260809 12:04:49.996562 13869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:04:49.998833 13869 webserver.cc:460] Webserver started at http://127.0.147.65:35523/ using document root <none> and password file <none>
I20260809 12:04:49.999504 13869 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:49.999697 13869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:50.000029 13869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:50.002787 13869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "97325524c0f748bf885cca97fa742d2c"
format_stamp: "Formatted at 2026-08-09 12:04:49 on dist-test-slave-sb2z"
server_key: "7be844738efe8ccd58a28e9c73415a2f"
server_key_iv: "1bad01b1f0a66d179cebd77b265c3eaa"
server_key_version: "encryptionkey@0"
I20260809 12:04:50.003650 13869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "97325524c0f748bf885cca97fa742d2c"
format_stamp: "Formatted at 2026-08-09 12:04:49 on dist-test-slave-sb2z"
server_key: "7be844738efe8ccd58a28e9c73415a2f"
server_key_iv: "1bad01b1f0a66d179cebd77b265c3eaa"
server_key_version: "encryptionkey@0"
I20260809 12:04:50.008749 13869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:04:50.012112 13890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:50.013371 13869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:04:50.013559 13869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "97325524c0f748bf885cca97fa742d2c"
format_stamp: "Formatted at 2026-08-09 12:04:49 on dist-test-slave-sb2z"
server_key: "7be844738efe8ccd58a28e9c73415a2f"
server_key_iv: "1bad01b1f0a66d179cebd77b265c3eaa"
server_key_version: "encryptionkey@0"
I20260809 12:04:50.013779 13869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:50.037427 13869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:50.038292 13869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:50.038616 13869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:50.039639 13869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:50.041134 13869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:50.041263 13869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:50.041406 13869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:50.041503 13869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:50.077728 13869 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:38597
I20260809 12:04:50.077817 14049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:38597 every 8 connection(s)
I20260809 12:04:50.079092 13869 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:04:50.086310 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 13869
I20260809 12:04:50.086520 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:04:50.087332 589 external_mini_cluster.cc:1428] Setting key 51c26e59a4d4a6e77288a4b6596b7005
I20260809 12:04:50.090322 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:41477
--builtin_ntp_servers=127.0.147.84:37111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 12:04:50.093118 14053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:50.081724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:54015 (local address 127.0.147.126:41477)
0809 12:04:50.082183 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:50.082188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:50.083437 (+ 1249us) server_negotiation.cc:407] Connection header received
0809 12:04:50.085271 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:50.085276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:50.085373 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:50.085495 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:50.086994 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:50.087896 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:50.089938 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:50.090619 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:50.090766 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:50.092134 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:50.092143 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:50.092152 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:50.092177 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:50.092234 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:50.092253 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:50.092347 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:50.092557 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 12:04:50.092665 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":242,"threads_started":1}
I20260809 12:04:50.093883 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:50.082369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:41477 (local address 127.0.147.65:54015)
0809 12:04:50.083273 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:50.083309 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:50.085002 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:50.085755 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:50.085769 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:50.085899 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:50.086771 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:50.086802 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:50.088086 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:50.088091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:50.089786 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:50.089798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:50.090825 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:50.090830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:50.091026 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:50.091820 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:04:50.091822 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:50.091867 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:04:50.091946 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:50.092361 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:50.092369 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:50.092375 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:04:50.092509 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 12:04:50.092792 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":264,"threads_started":1}
I20260809 12:04:50.095391 14050 heartbeater.cc:347] Connected to a master server at 127.0.147.126:41477
I20260809 12:04:50.095736 14050 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:50.096486 14050 heartbeater.cc:511] Master 127.0.147.126:41477 requested a full tablet report, sending...
I20260809 12:04:50.098477 13788 ts_manager.cc:194] Registered new tserver with Master: 97325524c0f748bf885cca97fa742d2c (127.0.147.65:38597)
I20260809 12:04:50.100051 13788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.65:54015
W20260809 12:04:50.428426 14057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:50.428766 14057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:50.428925 14057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:50.436005 14057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:50.436147 14057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:50.436305 14057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:04:50.445181 14057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:41477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:50.446442 14057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:50.448177 14057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:50.461748 14063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:50.461789 14068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:50.461704 14065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:50.462608 14057 server_base.cc:1034] running on GCE node
I20260809 12:04:50.463254 14057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:50.464188 14057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:50.465380 14057 hybrid_clock.cc:648] HybridClock initialized: now 1786277090465325 us; error 57 us; skew 500 ppm
I20260809 12:04:50.465654 14057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:04:50.467978 14057 webserver.cc:460] Webserver started at http://127.0.147.66:43887/ using document root <none> and password file <none>
I20260809 12:04:50.468611 14057 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:50.468698 14057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:50.468976 14057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:50.471909 14057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "4ebbe5c01f06439ba8bbed532a3c7957"
format_stamp: "Formatted at 2026-08-09 12:04:50 on dist-test-slave-sb2z"
server_key: "c7c456e56bae9a3fa4955e0961148d50"
server_key_iv: "d958dc032e1e4dcdedf07cabee55c200"
server_key_version: "encryptionkey@0"
I20260809 12:04:50.472729 14057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "4ebbe5c01f06439ba8bbed532a3c7957"
format_stamp: "Formatted at 2026-08-09 12:04:50 on dist-test-slave-sb2z"
server_key: "c7c456e56bae9a3fa4955e0961148d50"
server_key_iv: "d958dc032e1e4dcdedf07cabee55c200"
server_key_version: "encryptionkey@0"
I20260809 12:04:50.477885 14057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:04:50.481218 14076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:50.482496 14057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:04:50.482661 14057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "4ebbe5c01f06439ba8bbed532a3c7957"
format_stamp: "Formatted at 2026-08-09 12:04:50 on dist-test-slave-sb2z"
server_key: "c7c456e56bae9a3fa4955e0961148d50"
server_key_iv: "d958dc032e1e4dcdedf07cabee55c200"
server_key_version: "encryptionkey@0"
I20260809 12:04:50.482882 14057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:50.498656 14057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:50.499433 14057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:50.499748 14057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:50.500655 14057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:50.502076 14057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:50.502166 14057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:50.502311 14057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:50.502395 14057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:50.540482 14057 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:43813
I20260809 12:04:50.540583 14231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:43813 every 8 connection(s)
I20260809 12:04:50.541786 14057 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:04:50.550680 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 14057
I20260809 12:04:50.550848 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:04:50.551354 589 external_mini_cluster.cc:1428] Setting key edee7ccf4184b0158ebf74234b3ea77a
I20260809 12:04:50.554535 14053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:50.544400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:34349 (local address 127.0.147.126:41477)
0809 12:04:50.544582 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:50.544587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:50.545925 (+ 1338us) server_negotiation.cc:407] Connection header received
0809 12:04:50.547761 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:50.547766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:50.547857 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:50.547973 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:50.549002 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:50.549804 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:50.551470 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:50.551950 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:50.552083 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:50.553614 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:50.553620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:50.553624 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:50.553644 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:50.553696 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:50.553712 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:50.553815 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:50.554054 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 12:04:50.554151 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:04:50.554524 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:41477
--builtin_ntp_servers=127.0.147.84:37111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 12:04:50.555482 14235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:50.544889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:41477 (local address 127.0.147.66:34349)
0809 12:04:50.545767 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:50.545801 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:50.547440 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:50.548210 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:50.548217 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:50.548296 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:50.548831 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:50.548846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:50.549972 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:50.549976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:50.551329 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:50.551342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:50.552213 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:50.552222 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:50.552517 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:50.553316 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:04:50.553319 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:50.553360 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:04:50.553433 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:50.553831 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:50.553840 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:50.553846 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:04:50.553987 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 12:04:50.554331 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":272,"threads_started":1}
I20260809 12:04:50.556891 14232 heartbeater.cc:347] Connected to a master server at 127.0.147.126:41477
I20260809 12:04:50.557214 14232 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:50.558130 14232 heartbeater.cc:511] Master 127.0.147.126:41477 requested a full tablet report, sending...
I20260809 12:04:50.559628 13788 ts_manager.cc:194] Registered new tserver with Master: 4ebbe5c01f06439ba8bbed532a3c7957 (127.0.147.66:43813)
I20260809 12:04:50.560403 13788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.66:34349
W20260809 12:04:50.890429 14237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:50.890740 14237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:50.890904 14237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:50.898015 14237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:50.898164 14237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:50.898344 14237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:04:50.907397 14237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:41477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:50.908895 14237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:50.910888 14237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:50.924270 14250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:50.924310 14246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:50.924230 14245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:50.925295 14237 server_base.cc:1034] running on GCE node
I20260809 12:04:50.925915 14237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:50.926841 14237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:50.928098 14237 hybrid_clock.cc:648] HybridClock initialized: now 1786277090927973 us; error 100 us; skew 500 ppm
I20260809 12:04:50.928474 14237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:04:50.931452 14237 webserver.cc:460] Webserver started at http://127.0.147.67:33611/ using document root <none> and password file <none>
I20260809 12:04:50.932406 14237 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:50.932536 14237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:50.932951 14237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:50.936610 14237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "3390925a3cfc49ee85c48100769e76ff"
format_stamp: "Formatted at 2026-08-09 12:04:50 on dist-test-slave-sb2z"
server_key: "7e4f9d12eaab9b11a19b11d5bb1d04a8"
server_key_iv: "b65f84db34cd6384c11d5b7a9f775541"
server_key_version: "encryptionkey@0"
I20260809 12:04:50.937569 14237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "3390925a3cfc49ee85c48100769e76ff"
format_stamp: "Formatted at 2026-08-09 12:04:50 on dist-test-slave-sb2z"
server_key: "7e4f9d12eaab9b11a19b11d5bb1d04a8"
server_key_iv: "b65f84db34cd6384c11d5b7a9f775541"
server_key_version: "encryptionkey@0"
I20260809 12:04:50.943086 14237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:04:50.946547 14258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:50.947883 14237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:04:50.948071 14237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "3390925a3cfc49ee85c48100769e76ff"
format_stamp: "Formatted at 2026-08-09 12:04:50 on dist-test-slave-sb2z"
server_key: "7e4f9d12eaab9b11a19b11d5bb1d04a8"
server_key_iv: "b65f84db34cd6384c11d5b7a9f775541"
server_key_version: "encryptionkey@0"
I20260809 12:04:50.948283 14237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:50.980794 14237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:50.981706 14237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:50.982018 14237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:50.982982 14237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:50.984588 14237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:50.984719 14237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:50.984848 14237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:50.984937 14237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:51.019227 14237 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:41101
I20260809 12:04:51.019296 14427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:41101 every 8 connection(s)
I20260809 12:04:51.020587 14237 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:04:51.024855 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 14237
I20260809 12:04:51.025063 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:04:51.025549 589 external_mini_cluster.cc:1428] Setting key 5465b738c081b13b8bb13bff91372e82
I20260809 12:04:51.033685 14053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:51.023220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:55161 (local address 127.0.147.126:41477)
0809 12:04:51.023413 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:51.023418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:51.024863 (+ 1445us) server_negotiation.cc:407] Connection header received
0809 12:04:51.027025 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:51.027029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:51.027126 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:51.027262 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:51.028471 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.029208 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.030748 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.031173 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.031315 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:51.032804 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:51.032809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:51.032813 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:51.032830 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:51.032861 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:51.032874 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:51.032975 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:51.033234 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 12:04:51.033328 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:04:51.034600 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:51.023769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:41477 (local address 127.0.147.67:55161)
0809 12:04:51.024688 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:51.024737 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:51.026700 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:51.027635 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:51.027643 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:51.027726 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:51.028292 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:51.028308 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.029360 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.029366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:51.030591 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:51.030603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.031410 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.031417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:51.031734 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:51.032473 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:04:51.032476 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:51.032512 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:04:51.032582 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:51.032976 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:51.032989 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:51.032999 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:04:51.033201 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 12:04:51.033583 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":276,"threads_started":1}
I20260809 12:04:51.035940 14430 heartbeater.cc:347] Connected to a master server at 127.0.147.126:41477
I20260809 12:04:51.036209 14430 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:51.036891 14430 heartbeater.cc:511] Master 127.0.147.126:41477 requested a full tablet report, sending...
I20260809 12:04:51.038209 13788 ts_manager.cc:194] Registered new tserver with Master: 3390925a3cfc49ee85c48100769e76ff (127.0.147.67:41101)
I20260809 12:04:51.038949 13788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.67:55161
I20260809 12:04:51.041028 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 12:04:51.047124 14053 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:51.045572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.0.147.126:41477)
0809 12:04:51.045752 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:51.045756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:51.046301 (+ 545us) server_negotiation.cc:407] Connection header received
0809 12:04:51.046433 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:51.046436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:51.046510 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:51.046591 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:51.046939 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56398: BlockingRecv error: recv got EOF from 127.0.0.1:56398 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:04:51.047991 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 13869
I20260809 12:04:51.056080 14447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:51.049154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50444 (local address 127.0.147.65:38597)
0809 12:04:51.049767 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:51.049774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:51.049799 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:04:51.049852 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:51.049858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:51.050069 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:51.050247 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:51.050517 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.051635 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.052928 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.053551 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.053663 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:51.054674 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:51.054682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:51.054693 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:51.055327 (+ 634us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:51.055374 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:51.055393 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:51.055484 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:51.055672 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 12:04:51.055782 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":376,"threads_started":1}
I20260809 12:04:51.057286 14043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:51.138751 14050 heartbeater.cc:503] Master 127.0.147.126:41477 was elected leader, sending a full tablet report...
I20260809 12:04:51.140096 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 13869
W20260809 12:04:51.164738 13767 connection.cc:535] server connection from 127.0.147.65:54015 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 12:04:51.165093 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 14057
I20260809 12:04:51.172873 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:51.165998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41342 (local address 127.0.147.66:43813)
0809 12:04:51.166565 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:51.166573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:51.166600 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:04:51.166662 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:51.166668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:51.166901 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:51.167028 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:51.167390 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.168547 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.169771 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.170309 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.170401 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:51.171342 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:51.171351 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:51.171364 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:51.172043 (+ 679us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:51.172089 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:51.172108 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:51.172207 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:51.172347 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 12:04:51.172475 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":293,"threads_started":1}
I20260809 12:04:51.174132 14227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:51.255920 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 14057
I20260809 12:04:51.281533 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 14237
I20260809 12:04:51.289454 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:51.282596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34412 (local address 127.0.147.67:41101)
0809 12:04:51.283231 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:51.283239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:51.283269 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:04:51.283328 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:51.283333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:51.283621 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:51.283756 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:51.284165 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.285200 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.286365 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:51.286908 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:51.287005 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:51.288121 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:51.288128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:51.288141 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:51.288734 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:51.288791 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:51.288810 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:51.288899 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:51.289028 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 12:04:51.289142 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":328,"threads_started":1}
I20260809 12:04:51.290699 14422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:51.377198 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 14237
I20260809 12:04:51.404431 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 13745
I20260809 12:04:51.405689 13833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:51.511179 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 13745
2026-08-09T12:04:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2525 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T12:04:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:04:51Z Disabled control of system clock
I20260809 12:04:51.604835 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:32987
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:32987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 12:04:51.939682 14462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:51.939980 14462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:51.940152 14462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:51.947078 14462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:51.947201 14462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:51.947292 14462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:51.947397 14462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:51.947481 14462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:51.956372 14462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:32987
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:32987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:51.957615 14462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:51.959539 14462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:51.971235 14475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:51.971385 14471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:51.972069 14469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:51.972461 14462 server_base.cc:1034] running on GCE node
I20260809 12:04:51.973250 14462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:51.974643 14462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:51.975878 14462 hybrid_clock.cc:648] HybridClock initialized: now 1786277091975847 us; error 43 us; skew 500 ppm
I20260809 12:04:51.976228 14462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:04:51.978467 14462 webserver.cc:460] Webserver started at http://127.0.147.126:40325/ using document root <none> and password file <none>
I20260809 12:04:51.979156 14462 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:51.979290 14462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:51.979691 14462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:51.982532 14462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "1d2afa4ce8f24306ab94b8d530bbea05"
format_stamp: "Formatted at 2026-08-09 12:04:51 on dist-test-slave-sb2z"
server_key: "69f45d0fabc22ac867ccde8ec04261b1"
server_key_iv: "b75637e710eb49df0082459ff4fe8ae4"
server_key_version: "encryptionkey@0"
I20260809 12:04:51.983335 14462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "1d2afa4ce8f24306ab94b8d530bbea05"
format_stamp: "Formatted at 2026-08-09 12:04:51 on dist-test-slave-sb2z"
server_key: "69f45d0fabc22ac867ccde8ec04261b1"
server_key_iv: "b75637e710eb49df0082459ff4fe8ae4"
server_key_version: "encryptionkey@0"
I20260809 12:04:51.988423 14462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 12:04:51.991765 14485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:51.993059 14462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:04:51.993245 14462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "1d2afa4ce8f24306ab94b8d530bbea05"
format_stamp: "Formatted at 2026-08-09 12:04:51 on dist-test-slave-sb2z"
server_key: "69f45d0fabc22ac867ccde8ec04261b1"
server_key_iv: "b75637e710eb49df0082459ff4fe8ae4"
server_key_version: "encryptionkey@0"
I20260809 12:04:51.993484 14462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:51.999167 14462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:51.999967 14462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:52.000268 14462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:52.020128 14462 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:32987
I20260809 12:04:52.020228 14560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:32987 every 8 connection(s)
I20260809 12:04:52.021574 14462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:04:52.025453 14563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:52.030289 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 14462
I20260809 12:04:52.030509 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:04:52.031396 589 external_mini_cluster.cc:1428] Setting key 43de772581e800e24de6f4a4ea684b9b
I20260809 12:04:52.040486 14563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05: Bootstrap starting.
I20260809 12:04:52.040609 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:52.033675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57934 (local address 127.0.147.126:32987)
0809 12:04:52.034695 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:52.034707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:52.034746 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:04:52.036488 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:52.036512 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:52.036948 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:52.037378 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:52.038502 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:52.038508 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:52.038521 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:52.038534 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:52.038586 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:52.038600 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:52.038823 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:52.038952 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 12:04:52.039218 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":373,"threads_started":1}
I20260809 12:04:52.044046 14563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:52.044978 14563 log.cc:824] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:52.047479 14563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05: No bootstrap required, opened a new log
I20260809 12:04:52.054638 14563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2afa4ce8f24306ab94b8d530bbea05" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 32987 } }
I20260809 12:04:52.054961 14563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:52.055102 14563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2afa4ce8f24306ab94b8d530bbea05, State: Initialized, Role: FOLLOWER
I20260809 12:04:52.055771 14563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2afa4ce8f24306ab94b8d530bbea05" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 32987 } }
I20260809 12:04:52.056115 14563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:04:52.056260 14563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:04:52.056447 14563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:52.058607 14563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2afa4ce8f24306ab94b8d530bbea05" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 32987 } }
I20260809 12:04:52.059170 14563 leader_election.cc:302] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d2afa4ce8f24306ab94b8d530bbea05; no voters:
I20260809 12:04:52.059813 14563 leader_election.cc:288] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:04:52.059947 14572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:52.060729 14572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [term 1 LEADER]: Becoming Leader. State: Replica: 1d2afa4ce8f24306ab94b8d530bbea05, State: Running, Role: LEADER
I20260809 12:04:52.061269 14572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2afa4ce8f24306ab94b8d530bbea05" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 32987 } }
I20260809 12:04:52.062000 14563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:52.064101 14573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d2afa4ce8f24306ab94b8d530bbea05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2afa4ce8f24306ab94b8d530bbea05" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 32987 } } }
I20260809 12:04:52.064548 14573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:52.065052 14575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d2afa4ce8f24306ab94b8d530bbea05. Latest consensus state: current_term: 1 leader_uuid: "1d2afa4ce8f24306ab94b8d530bbea05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2afa4ce8f24306ab94b8d530bbea05" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 32987 } } }
I20260809 12:04:52.065411 14575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05 [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:52.065614 14584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:52.071626 14584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:52.085767 14584 catalog_manager.cc:1347] Generated new cluster ID: 112979f8ff4b4d5fbd7d8f2dad47ce37
I20260809 12:04:52.085884 14584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:52.100036 14584 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:04:52.100471 14584 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:52.110642 14584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d2afa4ce8f24306ab94b8d530bbea05: Generated new TSK 0
I20260809 12:04:52.111419 14584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:04:52.120610 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:32987
--builtin_ntp_servers=127.0.147.84:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 12:04:52.472879 14607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:52.473163 14607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:52.473316 14607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:52.480618 14607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:52.480731 14607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:52.480932 14607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:04:52.490464 14607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:32987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:52.491863 14607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:52.493630 14607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:52.507174 14616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:52.507301 14618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:52.507172 14615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:52.508083 14607 server_base.cc:1034] running on GCE node
I20260809 12:04:52.508644 14607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:52.509549 14607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:52.510787 14607 hybrid_clock.cc:648] HybridClock initialized: now 1786277092510738 us; error 57 us; skew 500 ppm
I20260809 12:04:52.511119 14607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:04:52.513447 14607 webserver.cc:460] Webserver started at http://127.0.147.65:42543/ using document root <none> and password file <none>
I20260809 12:04:52.514140 14607 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:52.514282 14607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:52.514612 14607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:52.517545 14607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "3d7cdfd718bd4fd2addd063ea570f544"
format_stamp: "Formatted at 2026-08-09 12:04:52 on dist-test-slave-sb2z"
server_key: "bc85af76b844f17e26016b729e1d3238"
server_key_iv: "63ff6810c6d09200f83a34c826adc97a"
server_key_version: "encryptionkey@0"
I20260809 12:04:52.518354 14607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "3d7cdfd718bd4fd2addd063ea570f544"
format_stamp: "Formatted at 2026-08-09 12:04:52 on dist-test-slave-sb2z"
server_key: "bc85af76b844f17e26016b729e1d3238"
server_key_iv: "63ff6810c6d09200f83a34c826adc97a"
server_key_version: "encryptionkey@0"
I20260809 12:04:52.523963 14607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:04:52.527454 14629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:52.528816 14607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:04:52.529012 14607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "3d7cdfd718bd4fd2addd063ea570f544"
format_stamp: "Formatted at 2026-08-09 12:04:52 on dist-test-slave-sb2z"
server_key: "bc85af76b844f17e26016b729e1d3238"
server_key_iv: "63ff6810c6d09200f83a34c826adc97a"
server_key_version: "encryptionkey@0"
I20260809 12:04:52.529278 14607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:52.535197 14607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:52.536151 14607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:52.536446 14607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:52.537418 14607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:52.538861 14607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:52.539000 14607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:52.539141 14607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:52.539242 14607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:52.574609 14607 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:40589
I20260809 12:04:52.574662 14792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:40589 every 8 connection(s)
I20260809 12:04:52.576097 14607 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:04:52.584326 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:52.578611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:37835 (local address 127.0.147.126:32987)
0809 12:04:52.579091 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:52.579096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:52.580291 (+ 1195us) server_negotiation.cc:407] Connection header received
0809 12:04:52.582026 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:52.582032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:52.582108 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:52.582228 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:52.583496 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:52.583503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:52.583508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:52.583550 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:52.583611 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:52.583627 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:52.583770 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:52.583960 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 12:04:52.584031 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":264,"threads_started":1}
I20260809 12:04:52.585206 14800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:52.579179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:32987 (local address 127.0.147.65:37835)
0809 12:04:52.580127 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:52.580167 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:52.581754 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:52.582469 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:52.582477 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:52.582567 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:52.583202 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:04:52.583206 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:52.583243 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:04:52.583324 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:52.583735 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:52.583749 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:52.583757 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:04:52.584050 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 12:04:52.584312 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":344,"threads_started":1}
I20260809 12:04:52.586114 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 14607
I20260809 12:04:52.586340 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:04:52.586745 14795 heartbeater.cc:347] Connected to a master server at 127.0.147.126:32987
I20260809 12:04:52.587044 14795 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:52.587158 589 external_mini_cluster.cc:1428] Setting key 96af855c926edb540c2b4158b4371812
I20260809 12:04:52.587718 14795 heartbeater.cc:511] Master 127.0.147.126:32987 requested a full tablet report, sending...
I20260809 12:04:52.589776 14511 ts_manager.cc:194] Registered new tserver with Master: 3d7cdfd718bd4fd2addd063ea570f544 (127.0.147.65:40589)
I20260809 12:04:52.590112 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:32987
--builtin_ntp_servers=127.0.147.84:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 12:04:52.929093 14801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:52.929355 14801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:52.929553 14801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:52.937096 14801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:52.937212 14801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:52.937393 14801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:04:52.946529 14801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:32987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:52.947794 14801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:52.949512 14801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:52.962395 14809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:52.962337 14808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:52.962599 14812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:52.963773 14801 server_base.cc:1034] running on GCE node
I20260809 12:04:52.964422 14801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:52.965296 14801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:52.966497 14801 hybrid_clock.cc:648] HybridClock initialized: now 1786277092966460 us; error 26 us; skew 500 ppm
I20260809 12:04:52.966822 14801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:04:52.969151 14801 webserver.cc:460] Webserver started at http://127.0.147.66:43421/ using document root <none> and password file <none>
I20260809 12:04:52.969795 14801 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:52.969880 14801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:52.970244 14801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:52.973085 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "191ee7736ff149ceb6d8bbcb40822522"
format_stamp: "Formatted at 2026-08-09 12:04:52 on dist-test-slave-sb2z"
server_key: "98d27d02428de0a860e8f6b402c91c1f"
server_key_iv: "563d0cde1b4548d67dafafa698bfcd34"
server_key_version: "encryptionkey@0"
I20260809 12:04:52.973878 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "191ee7736ff149ceb6d8bbcb40822522"
format_stamp: "Formatted at 2026-08-09 12:04:52 on dist-test-slave-sb2z"
server_key: "98d27d02428de0a860e8f6b402c91c1f"
server_key_iv: "563d0cde1b4548d67dafafa698bfcd34"
server_key_version: "encryptionkey@0"
I20260809 12:04:52.978680 14801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260809 12:04:52.981997 14819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:52.983259 14801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:04:52.983385 14801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "191ee7736ff149ceb6d8bbcb40822522"
format_stamp: "Formatted at 2026-08-09 12:04:52 on dist-test-slave-sb2z"
server_key: "98d27d02428de0a860e8f6b402c91c1f"
server_key_iv: "563d0cde1b4548d67dafafa698bfcd34"
server_key_version: "encryptionkey@0"
I20260809 12:04:52.983649 14801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:52.989037 14801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:52.989742 14801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:52.989966 14801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:52.990765 14801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:52.992084 14801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:52.992172 14801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:52.992250 14801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:52.992306 14801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:53.026595 14801 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:39515
I20260809 12:04:53.026691 14985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:39515 every 8 connection(s)
I20260809 12:04:53.028183 14801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:04:53.030090 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 14801
I20260809 12:04:53.030305 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:04:53.030748 589 external_mini_cluster.cc:1428] Setting key b2f8572868a7ca824ac2dc9e28e33635
I20260809 12:04:53.034420 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:32987
--builtin_ntp_servers=127.0.147.84:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 12:04:53.039234 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:53.031658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:33067 (local address 127.0.147.126:32987)
0809 12:04:53.031845 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:53.031851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:53.033837 (+ 1986us) server_negotiation.cc:407] Connection header received
0809 12:04:53.036683 (+ 2846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:53.036689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:53.036751 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:53.036857 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:53.038457 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:53.038464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:53.038471 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:53.038493 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:53.038532 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:53.038546 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:53.038661 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:53.038878 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 12:04:53.038939 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:04:53.040429 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:53.031408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:32987 (local address 127.0.147.66:33067)
0809 12:04:53.033685 (+ 2277us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:53.033734 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:53.036368 (+ 2634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:53.037131 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:53.037146 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:53.037271 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:53.038074 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:04:53.038080 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:53.038132 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:04:53.038248 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:53.038657 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:53.038670 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:53.038679 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 12:04:53.038959 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 12:04:53.039213 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1711,"thread_start_us":1357,"threads_started":1}
I20260809 12:04:53.041839 14986 heartbeater.cc:347] Connected to a master server at 127.0.147.126:32987
I20260809 12:04:53.042155 14986 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:53.042742 14986 heartbeater.cc:511] Master 127.0.147.126:32987 requested a full tablet report, sending...
I20260809 12:04:53.044147 14511 ts_manager.cc:194] Registered new tserver with Master: 191ee7736ff149ceb6d8bbcb40822522 (127.0.147.66:39515)
W20260809 12:04:53.382591 14989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:53.382858 14989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:53.383049 14989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:53.390424 14989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:53.390547 14989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:53.390739 14989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:04:53.399863 14989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:32987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:53.401154 14989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:53.402894 14989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:53.415838 14994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:53.415863 14999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:53.415824 15001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:53.417088 14989 server_base.cc:1034] running on GCE node
I20260809 12:04:53.417675 14989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:53.418566 14989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:53.419790 14989 hybrid_clock.cc:648] HybridClock initialized: now 1786277093419744 us; error 61 us; skew 500 ppm
I20260809 12:04:53.420116 14989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:04:53.426858 14989 webserver.cc:460] Webserver started at http://127.0.147.67:34323/ using document root <none> and password file <none>
I20260809 12:04:53.427645 14989 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:53.427783 14989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:53.428123 14989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:53.430990 14989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "f95ad85c7fd54f928b25016e25728286"
format_stamp: "Formatted at 2026-08-09 12:04:53 on dist-test-slave-sb2z"
server_key: "f165a63efa5420c63e7bd4fe66b07e9f"
server_key_iv: "0bc9e9bc7e0a67fe2390626c56e6d9a2"
server_key_version: "encryptionkey@0"
I20260809 12:04:53.431941 14989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "f95ad85c7fd54f928b25016e25728286"
format_stamp: "Formatted at 2026-08-09 12:04:53 on dist-test-slave-sb2z"
server_key: "f165a63efa5420c63e7bd4fe66b07e9f"
server_key_iv: "0bc9e9bc7e0a67fe2390626c56e6d9a2"
server_key_version: "encryptionkey@0"
I20260809 12:04:53.437454 14989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:04:53.440985 15009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:53.442286 14989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:04:53.442482 14989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "f95ad85c7fd54f928b25016e25728286"
format_stamp: "Formatted at 2026-08-09 12:04:53 on dist-test-slave-sb2z"
server_key: "f165a63efa5420c63e7bd4fe66b07e9f"
server_key_iv: "0bc9e9bc7e0a67fe2390626c56e6d9a2"
server_key_version: "encryptionkey@0"
I20260809 12:04:53.442727 14989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:53.448663 14989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:53.449460 14989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:53.449754 14989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:53.450690 14989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:53.452112 14989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:53.452231 14989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:53.452394 14989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:53.452486 14989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:53.486764 14989 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:43765
I20260809 12:04:53.486855 15172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:43765 every 8 connection(s)
I20260809 12:04:53.488235 14989 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:04:53.494745 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 14989
I20260809 12:04:53.494961 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:04:53.495573 589 external_mini_cluster.cc:1428] Setting key db4f8c14d07e0aec1451fed44c9a54b5
I20260809 12:04:53.496650 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:53.490756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:39715 (local address 127.0.147.126:32987)
0809 12:04:53.490986 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:53.490993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:53.492426 (+ 1433us) server_negotiation.cc:407] Connection header received
0809 12:04:53.494064 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:53.494069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:53.494124 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:53.494216 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:53.495897 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:53.495906 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:53.495911 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:53.495929 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:53.495970 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:53.495983 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:53.496082 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:53.496322 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 12:04:53.496390 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:04:53.497768 15178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:53.491313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:32987 (local address 127.0.147.67:39715)
0809 12:04:53.492262 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:53.492304 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:53.493809 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:53.494473 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:53.494487 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:53.494633 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:53.495511 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:04:53.495514 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:53.495605 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:04:53.495723 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:53.496090 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:53.496102 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:53.496110 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:04:53.496344 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 12:04:53.496617 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":339,"threads_started":1}
I20260809 12:04:53.499513 15174 heartbeater.cc:347] Connected to a master server at 127.0.147.126:32987
I20260809 12:04:53.499843 15174 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:53.500403 15174 heartbeater.cc:511] Master 127.0.147.126:32987 requested a full tablet report, sending...
I20260809 12:04:53.501715 14511 ts_manager.cc:194] Registered new tserver with Master: f95ad85c7fd54f928b25016e25728286 (127.0.147.67:43765)
I20260809 12:04:53.511034 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 12:04:53.517221 14799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:04:53.515632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.0.147.126:32987)
0809 12:04:53.515826 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:53.515834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:53.516327 (+ 493us) server_negotiation.cc:407] Connection header received
0809 12:04:53.516469 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:53.516473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:53.516525 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:53.516597 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:53.516976 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57948: BlockingRecv error: recv got EOF from 127.0.0.1:57948 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:04:53.517717 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 14607
I20260809 12:04:53.521376 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:53.518806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.0.147.65:40589)
0809 12:04:53.519343 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:53.519349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:53.519374 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:04:53.519430 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:53.519435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:53.519682 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:53.519803 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:53.520690 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:53.520696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:53.520707 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:53.520719 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:53.520764 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:53.520782 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:53.520889 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:53.520974 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 12:04:53.521104 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":264,"threads_started":1}
I20260809 12:04:53.522548 14785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:53.603621 14795 heartbeater.cc:503] Master 127.0.147.126:32987 was elected leader, sending a full tablet report...
I20260809 12:04:53.605451 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 14607
I20260809 12:04:53.628981 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 14801
I20260809 12:04:53.632719 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:53.630056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.0.147.66:39515)
0809 12:04:53.630586 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:53.630593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:53.630620 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:04:53.630682 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:53.630689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:53.630897 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:53.631023 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:53.631947 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:53.631954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:53.631965 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:53.631985 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:53.632030 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:53.632045 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:53.632151 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:53.632283 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 12:04:53.632392 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":257,"threads_started":1}
I20260809 12:04:53.633955 14979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:53.716169 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 14801
I20260809 12:04:53.741274 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 14989
I20260809 12:04:53.745478 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:53.742252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50930 (local address 127.0.147.67:43765)
0809 12:04:53.742933 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:53.742940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:53.742975 (+ 35us) server_negotiation.cc:407] Connection header received
0809 12:04:53.743044 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:53.743050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:53.743264 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:53.743446 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:53.744685 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:53.744692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:53.744703 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:04:53.744720 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:53.744760 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:04:53.744773 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:53.744977 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:53.745060 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 12:04:53.745177 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":355,"threads_started":1}
I20260809 12:04:53.746776 15168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:53.828892 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 14989
I20260809 12:04:53.851298 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 14462
I20260809 12:04:53.852682 14553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:53.955359 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 14462
2026-08-09T12:04:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2446 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:04:54 dist-test-slave-sb2z krb5kdc[15202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:04:54 dist-test-slave-sb2z krb5kdc[15202](info): set up 2 sockets
Aug 09 12:04:54 dist-test-slave-sb2z krb5kdc[15202](info): commencing operation
krb5kdc: starting...
W20260809 12:04:55.262523 589 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:04:55 dist-test-slave-sb2z krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:04:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:04:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:04:55.575060 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45005
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:42485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:45005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:55.913403 15227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:55.913686 15227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:55.913893 15227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:55.921028 15227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:04:55.921129 15227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:55.921214 15227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:55.921336 15227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:04:55.921433 15227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:04:55.931062 15227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:42485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:45005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:55.932430 15227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:55.933743 15227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:55.946089 15241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:55.946273 15235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:55.947088 15236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:55.952116 15227 server_base.cc:1034] running on GCE node
I20260809 12:04:55.952919 15227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:55.955015 15227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:55.956269 15227 hybrid_clock.cc:648] HybridClock initialized: now 1786277095956192 us; error 50 us; skew 500 ppm
I20260809 12:04:55.956743 15227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:55 dist-test-slave-sb2z krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277095, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:55.960631 15227 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:04:55.962626 15227 webserver.cc:460] Webserver started at http://127.0.147.126:46789/ using document root <none> and password file <none>
I20260809 12:04:55.963661 15227 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:55.963830 15227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:55.964241 15227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:55.968176 15227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "2e1921d401a9498c97c24e67f5c8b1bf"
format_stamp: "Formatted at 2026-08-09 12:04:55 on dist-test-slave-sb2z"
server_key: "c050e9be21916a62de27dcd91b8597b5"
server_key_iv: "d6ea9b1d96813013a44556eae718465d"
server_key_version: "encryptionkey@0"
I20260809 12:04:55.969259 15227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "2e1921d401a9498c97c24e67f5c8b1bf"
format_stamp: "Formatted at 2026-08-09 12:04:55 on dist-test-slave-sb2z"
server_key: "c050e9be21916a62de27dcd91b8597b5"
server_key_iv: "d6ea9b1d96813013a44556eae718465d"
server_key_version: "encryptionkey@0"
I20260809 12:04:55.977278 15227 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260809 12:04:55.982331 15250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:55.984251 15227 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:04:55.984485 15227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "2e1921d401a9498c97c24e67f5c8b1bf"
format_stamp: "Formatted at 2026-08-09 12:04:55 on dist-test-slave-sb2z"
server_key: "c050e9be21916a62de27dcd91b8597b5"
server_key_iv: "d6ea9b1d96813013a44556eae718465d"
server_key_version: "encryptionkey@0"
I20260809 12:04:55.984755 15227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:56.006016 15227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:56.011487 15227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:56.011807 15227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:56.030813 15227 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:45005
I20260809 12:04:56.030846 15325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:45005 every 8 connection(s)
I20260809 12:04:56.032259 15227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:04:56.035948 15326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:56.036592 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 15227
I20260809 12:04:56.036849 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:04:56.037554 589 external_mini_cluster.cc:1428] Setting key ea7ac3940bbb4048f40df6f331afbd9f
I20260809 12:04:56.050428 15326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf: Bootstrap starting.
Aug 09 12:04:56 dist-test-slave-sb2z krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:56.054204 15326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:56.055269 15326 log.cc:824] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:56.058734 15326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf: No bootstrap required, opened a new log
I20260809 12:04:56.062492 15330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:56.039652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39924 (local address 127.0.147.126:45005)
0809 12:04:56.040679 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:56.040691 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:56.040740 (+ 49us) server_negotiation.cc:407] Connection header received
0809 12:04:56.042431 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:56.042460 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:56.042844 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:56.043193 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:56.045169 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:56.046461 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:56.048328 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:56.048962 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:56.049106 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:56.053928 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:56.053937 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:56.053951 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:56.053986 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:56.057874 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:56.058554 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:56.058563 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:56.058571 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:56.058821 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:56.059244 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:56.059248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:56.059251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:56.059995 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:56.060329 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:56.060636 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 12:04:56.061124 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":578,"threads_started":1}
I20260809 12:04:56.068430 15326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1921d401a9498c97c24e67f5c8b1bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45005 } }
I20260809 12:04:56.068758 15326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:56.068832 15326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1921d401a9498c97c24e67f5c8b1bf, State: Initialized, Role: FOLLOWER
I20260809 12:04:56.069365 15326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1921d401a9498c97c24e67f5c8b1bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45005 } }
I20260809 12:04:56.069577 15326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:04:56.069675 15326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:04:56.069801 15326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:56.072181 15326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1921d401a9498c97c24e67f5c8b1bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45005 } }
I20260809 12:04:56.072628 15326 leader_election.cc:302] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e1921d401a9498c97c24e67f5c8b1bf; no voters:
I20260809 12:04:56.073181 15326 leader_election.cc:288] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:04:56.073330 15333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:56.074051 15333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [term 1 LEADER]: Becoming Leader. State: Replica: 2e1921d401a9498c97c24e67f5c8b1bf, State: Running, Role: LEADER
I20260809 12:04:56.074587 15333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1921d401a9498c97c24e67f5c8b1bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45005 } }
I20260809 12:04:56.075297 15326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:04:56.078544 15335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e1921d401a9498c97c24e67f5c8b1bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1921d401a9498c97c24e67f5c8b1bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45005 } } }
I20260809 12:04:56.078886 15335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:56.078641 15337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e1921d401a9498c97c24e67f5c8b1bf. Latest consensus state: current_term: 1 leader_uuid: "2e1921d401a9498c97c24e67f5c8b1bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1921d401a9498c97c24e67f5c8b1bf" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45005 } } }
I20260809 12:04:56.079082 15337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf [sys.catalog]: This master's current role is: LEADER
I20260809 12:04:56.080678 15341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:04:56.085796 15341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:04:56.094467 15341 catalog_manager.cc:1347] Generated new cluster ID: 0dc6884adbc34ec7908fbeac54efa7c0
I20260809 12:04:56.094600 15341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:04:56.107056 15341 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:04:56.108487 15341 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:04:56.119565 15341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf: Generated new TSK 0
I20260809 12:04:56.120342 15341 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:04:56.197677 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45005
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:42485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:56.540832 15368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:56.541201 15368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:56.541378 15368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:56.548638 15368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:56.548791 15368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:56.548969 15368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:04:56.558869 15368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:42485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:56.560241 15368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:56.561556 15368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:56.574461 15377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:56.574528 15380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:56.574555 15376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:56.575577 15368 server_base.cc:1034] running on GCE node
I20260809 12:04:56.576210 15368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:56.577253 15368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:56.578491 15368 hybrid_clock.cc:648] HybridClock initialized: now 1786277096578445 us; error 29 us; skew 500 ppm
I20260809 12:04:56.578841 15368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:56 dist-test-slave-sb2z krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277096, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:56.582471 15368 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:04:56.584097 15368 webserver.cc:460] Webserver started at http://127.0.147.65:46147/ using document root <none> and password file <none>
I20260809 12:04:56.584808 15368 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:56.584957 15368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:56.585283 15368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:56.588285 15368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "e88e9df892c848fe8c4bb68b8d9dc842"
format_stamp: "Formatted at 2026-08-09 12:04:56 on dist-test-slave-sb2z"
server_key: "ddb32d8f83163cc9866106e12d8b0680"
server_key_iv: "a4c0985601093fe54eba83c8901d0356"
server_key_version: "encryptionkey@0"
I20260809 12:04:56.589058 15368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "e88e9df892c848fe8c4bb68b8d9dc842"
format_stamp: "Formatted at 2026-08-09 12:04:56 on dist-test-slave-sb2z"
server_key: "ddb32d8f83163cc9866106e12d8b0680"
server_key_iv: "a4c0985601093fe54eba83c8901d0356"
server_key_version: "encryptionkey@0"
I20260809 12:04:56.594765 15368 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:04:56.598284 15390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:56.599695 15368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:04:56.599875 15368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "e88e9df892c848fe8c4bb68b8d9dc842"
format_stamp: "Formatted at 2026-08-09 12:04:56 on dist-test-slave-sb2z"
server_key: "ddb32d8f83163cc9866106e12d8b0680"
server_key_iv: "a4c0985601093fe54eba83c8901d0356"
server_key_version: "encryptionkey@0"
I20260809 12:04:56.600076 15368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:56.624069 15368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:56.628120 15368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:56.628460 15368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:56.629431 15368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:56.630779 15368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:56.630887 15368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:56.631035 15368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:56.631110 15368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:56.665377 15368 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:33757
I20260809 12:04:56.665443 15567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:33757 every 8 connection(s)
I20260809 12:04:56.666712 15368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:04:56.668100 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 15368
I20260809 12:04:56.668351 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:04:56.669211 589 external_mini_cluster.cc:1428] Setting key f79907a5a93c16e3ac4b2ccb07a12caa
Aug 09 12:04:56 dist-test-slave-sb2z krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277096, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:56.687255 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:56.669711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:49227 (local address 127.0.147.126:45005)
0809 12:04:56.670232 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:56.670238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:56.671487 (+ 1249us) server_negotiation.cc:407] Connection header received
0809 12:04:56.673209 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:56.673214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:56.673301 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:56.673429 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:56.674928 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:56.675803 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:56.677365 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:56.677759 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:56.677897 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:56.681484 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:56.681489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:56.681493 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:56.681522 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:56.684211 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:56.684938 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:56.684945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:56.684950 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:56.685108 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:56.685522 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:56.685526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:56.685528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:56.685809 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:56.685978 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:56.686651 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 12:04:56.686883 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":261,"threads_started":1}
I20260809 12:04:56.688221 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:56.670266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45005 (local address 127.0.147.65:49227)
0809 12:04:56.671080 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:56.671164 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:56.672942 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:56.673683 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:56.673691 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:56.674206 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:56.674760 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:56.674775 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:56.675965 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:56.675970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:56.677204 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:56.677216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:56.677999 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:56.678009 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:56.678245 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:56.678917 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:56.678939 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:56.681278 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:56.684369 (+ 3091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:56.684375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:56.684387 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:56.684818 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:56.685223 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:56.685226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:56.685228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:56.685407 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:56.685961 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:56.685969 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:56.686282 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 12:04:56.686547 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 12:04:56.687047 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":309,"threads_started":1}
I20260809 12:04:56.689693 15569 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45005
I20260809 12:04:56.690017 15569 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:56.690776 15569 heartbeater.cc:511] Master 127.0.147.126:45005 requested a full tablet report, sending...
I20260809 12:04:56.692855 15273 ts_manager.cc:194] Registered new tserver with Master: e88e9df892c848fe8c4bb68b8d9dc842 (127.0.147.65:33757)
I20260809 12:04:56.694275 15273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:49227
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:04:56.736435 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45005
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:42485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:57.076594 15577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:57.076954 15577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:57.077147 15577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:57.084777 15577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:57.084882 15577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:57.085032 15577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:04:57.094964 15577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:42485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:57.096350 15577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:57.097659 15577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:57.110652 15587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:57.110702 15585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:57.110743 15584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:57.111641 15577 server_base.cc:1034] running on GCE node
I20260809 12:04:57.112246 15577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:57.113159 15577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:57.114398 15577 hybrid_clock.cc:648] HybridClock initialized: now 1786277097114357 us; error 48 us; skew 500 ppm
I20260809 12:04:57.114717 15577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:57 dist-test-slave-sb2z krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277097, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:57.118031 15577 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:04:57.119506 15577 webserver.cc:460] Webserver started at http://127.0.147.66:41361/ using document root <none> and password file <none>
I20260809 12:04:57.120281 15577 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:57.120445 15577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:57.120740 15577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:57.123659 15577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "de606321aafd4d17b31ce8439b261e0f"
format_stamp: "Formatted at 2026-08-09 12:04:57 on dist-test-slave-sb2z"
server_key: "3a425ba266034b3d13b2e88382a423c6"
server_key_iv: "3ae19389a6ca2cea75da197a523a7573"
server_key_version: "encryptionkey@0"
I20260809 12:04:57.124487 15577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "de606321aafd4d17b31ce8439b261e0f"
format_stamp: "Formatted at 2026-08-09 12:04:57 on dist-test-slave-sb2z"
server_key: "3a425ba266034b3d13b2e88382a423c6"
server_key_iv: "3ae19389a6ca2cea75da197a523a7573"
server_key_version: "encryptionkey@0"
I20260809 12:04:57.129492 15577 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:04:57.132905 15598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:57.134189 15577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 12:04:57.134361 15577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "de606321aafd4d17b31ce8439b261e0f"
format_stamp: "Formatted at 2026-08-09 12:04:57 on dist-test-slave-sb2z"
server_key: "3a425ba266034b3d13b2e88382a423c6"
server_key_iv: "3ae19389a6ca2cea75da197a523a7573"
server_key_version: "encryptionkey@0"
I20260809 12:04:57.134562 15577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:57.152292 15577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:57.156173 15577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:57.156456 15577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:57.157361 15577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:57.158886 15577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:57.158993 15577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:57.159152 15577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:57.159225 15577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:57.193897 15577 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:39687
I20260809 12:04:57.193953 15774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:39687 every 8 connection(s)
I20260809 12:04:57.195204 15577 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:04:57.205439 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 15577
I20260809 12:04:57.205657 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:04:57.206245 589 external_mini_cluster.cc:1428] Setting key 106871884c2961173998c2a9a88e09ec
Aug 09 12:04:57 dist-test-slave-sb2z krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277097, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:57.216948 15779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.197741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:52205 (local address 127.0.147.126:45005)
0809 12:04:57.198206 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:57.198213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:57.199235 (+ 1022us) server_negotiation.cc:407] Connection header received
0809 12:04:57.201103 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:57.201108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:57.201176 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:57.201275 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:57.202706 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.203502 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.205263 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.205764 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.205873 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.209839 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:57.209849 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:57.209857 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:57.209903 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:57.213614 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:57.214376 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:57.214385 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:57.214390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:57.214562 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:57.214961 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:57.214968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:57.214973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:57.215347 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:57.215506 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:57.216217 (+ 711us) server_negotiation.cc:299] Negotiation successful
0809 12:04:57.216477 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":217,"threads_started":1}
I20260809 12:04:57.217598 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.198351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45005 (local address 127.0.147.66:52205)
0809 12:04:57.199055 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:57.199107 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:57.200824 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:57.201472 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:57.201479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:57.201992 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:57.202551 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.202570 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.203727 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.203731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.205105 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.205121 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.205998 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.206005 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.206323 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.207052 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:57.207090 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:57.209607 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:57.213805 (+ 4198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:57.213812 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:57.213825 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:57.214261 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:57.214670 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:57.214673 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:57.214676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:57.214859 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:57.215473 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:57.215479 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:57.215854 (+ 375us) client_negotiation.cc:769] Sending connection context
0809 12:04:57.216087 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 12:04:57.216468 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":269,"threads_started":1}
I20260809 12:04:57.219071 15777 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45005
I20260809 12:04:57.219403 15777 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:57.220340 15777 heartbeater.cc:511] Master 127.0.147.126:45005 requested a full tablet report, sending...
I20260809 12:04:57.221902 15273 ts_manager.cc:194] Registered new tserver with Master: de606321aafd4d17b31ce8439b261e0f (127.0.147.66:39687)
I20260809 12:04:57.222743 15273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:52205
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:04:57.272595 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45005
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:42485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:04:57.615875 15784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:04:57.616195 15784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:04:57.616367 15784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:04:57.623672 15784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:04:57.623807 15784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:04:57.624039 15784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:04:57.633714 15784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:42485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:04:57.634982 15784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:04:57.636358 15784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:04:57.649645 15793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:57.649688 15791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:04:57.649605 15797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:04:57.650718 15784 server_base.cc:1034] running on GCE node
I20260809 12:04:57.651363 15784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:04:57.652334 15784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:04:57.653543 15784 hybrid_clock.cc:648] HybridClock initialized: now 1786277097653505 us; error 30 us; skew 500 ppm
I20260809 12:04:57.653878 15784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:04:57 dist-test-slave-sb2z krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277097, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:04:57.657083 15784 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:04:57.658531 15784 webserver.cc:460] Webserver started at http://127.0.147.67:46713/ using document root <none> and password file <none>
I20260809 12:04:57.659215 15784 fs_manager.cc:359] Metadata directory not provided
I20260809 12:04:57.659348 15784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:04:57.659724 15784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:04:57.662648 15784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "9ba6a75019b64b178675142070345bad"
format_stamp: "Formatted at 2026-08-09 12:04:57 on dist-test-slave-sb2z"
server_key: "10881661df54cb3d204a11b452578460"
server_key_iv: "23a17ba36dd2b1dc8779733fa609daa7"
server_key_version: "encryptionkey@0"
I20260809 12:04:57.663394 15784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "9ba6a75019b64b178675142070345bad"
format_stamp: "Formatted at 2026-08-09 12:04:57 on dist-test-slave-sb2z"
server_key: "10881661df54cb3d204a11b452578460"
server_key_iv: "23a17ba36dd2b1dc8779733fa609daa7"
server_key_version: "encryptionkey@0"
I20260809 12:04:57.668722 15784 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:04:57.672158 15806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:57.673497 15784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:04:57.673677 15784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "9ba6a75019b64b178675142070345bad"
format_stamp: "Formatted at 2026-08-09 12:04:57 on dist-test-slave-sb2z"
server_key: "10881661df54cb3d204a11b452578460"
server_key_iv: "23a17ba36dd2b1dc8779733fa609daa7"
server_key_version: "encryptionkey@0"
I20260809 12:04:57.673895 15784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:04:57.691694 15784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:04:57.695660 15784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:04:57.695961 15784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:04:57.696960 15784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:04:57.697088 15569 heartbeater.cc:503] Master 127.0.147.126:45005 was elected leader, sending a full tablet report...
I20260809 12:04:57.698660 15784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:04:57.698771 15784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:57.698920 15784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:04:57.698998 15784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:04:57.736541 15784 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:45987
I20260809 12:04:57.736590 15965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:45987 every 8 connection(s)
I20260809 12:04:57.738068 15784 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:04:57.742250 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 15784
I20260809 12:04:57.742551 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:04:57.743502 589 external_mini_cluster.cc:1428] Setting key 3aa23c4bf57ee1170a603b9e787dae4a
Aug 09 12:04:57 dist-test-slave-sb2z krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277097, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:04:57.760468 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.740828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:56457 (local address 127.0.147.126:45005)
0809 12:04:57.741403 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:57.741408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:57.743621 (+ 2213us) server_negotiation.cc:407] Connection header received
0809 12:04:57.745618 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:57.745624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:57.745732 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:57.745845 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:57.747844 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.748619 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.750390 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.750790 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.750882 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.754245 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:57.754250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:57.754254 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:57.754285 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:57.756969 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:57.757896 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:57.757902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:57.757906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:57.758066 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:57.758637 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:57.758644 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:57.758650 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:57.759024 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:57.759193 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:57.760012 (+ 819us) server_negotiation.cc:299] Negotiation successful
0809 12:04:57.760192 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":279,"threads_started":1}
I20260809 12:04:57.761421 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.741452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45005 (local address 127.0.147.67:56457)
0809 12:04:57.742448 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:57.742580 (+ 132us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:57.745205 (+ 2625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:57.746139 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:57.746148 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:57.746892 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:04:57.747684 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.747705 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.748772 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.748776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.750259 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.750272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.750993 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.751000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.751190 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.751865 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:04:57.751893 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:04:57.754046 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:04:57.757094 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:57.757100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:57.757111 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:57.757739 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:57.758211 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:04:57.758219 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:04:57.758222 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:04:57.758477 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:04:57.759174 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:04:57.759182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:04:57.759651 (+ 469us) client_negotiation.cc:769] Sending connection context
0809 12:04:57.759896 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 12:04:57.760296 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":422,"threads_started":1}
I20260809 12:04:57.762874 15967 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45005
I20260809 12:04:57.763236 15967 heartbeater.cc:464] Registering TS with master...
I20260809 12:04:57.764092 15967 heartbeater.cc:511] Master 127.0.147.126:45005 requested a full tablet report, sending...
I20260809 12:04:57.765589 15273 ts_manager.cc:194] Registered new tserver with Master: 9ba6a75019b64b178675142070345bad (127.0.147.67:45987)
I20260809 12:04:57.766395 15273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:56457
I20260809 12:04:57.771812 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:04:57.788666 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.776487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.0.147.126:45005)
0809 12:04:57.776692 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:57.776700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:57.777241 (+ 541us) server_negotiation.cc:407] Connection header received
0809 12:04:57.777417 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:57.777420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:57.777482 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:57.777563 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:57.778968 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.779759 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.781016 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.781415 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.781524 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.783878 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:57.783882 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:57.783885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:57.783912 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:57.786340 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:57.786940 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:57.786944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:57.786946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:57.787062 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:57.787365 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:57.787368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:57.787370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:57.787731 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:57.787914 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:57.788166 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 12:04:57.788320 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:04:57.792752 15273 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:04:57.799666 15273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:04:57.824491 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.811242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:33757 (local address 127.0.0.1:45396)
0809 12:04:57.812196 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:57.812253 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:57.812482 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:57.813892 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:57.813904 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:57.813945 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:57.814429 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.814445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.817262 (+ 2817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.817271 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.820481 (+ 3210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.820503 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.823111 (+ 2608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.823121 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.823412 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.823491 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 12:04:57.823660 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 12:04:57.823944 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":932,"threads_started":1}
I20260809 12:04:57.826591 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.811431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45396 (local address 127.0.147.65:33757)
0809 12:04:57.813183 (+ 1752us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:57.813206 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:57.813237 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:04:57.813340 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:57.813351 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:57.813617 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:57.814725 (+ 1108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:57.815004 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.817069 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.820783 (+ 3714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.822818 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.825468 (+ 2650us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.825817 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:57.825912 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 12:04:57.826132 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1604,"thread_start_us":1455,"threads_started":1}
I20260809 12:04:57.832808 15456 tablet_service.cc:1467] Processing CreateTablet for tablet 0879bcc51add439583920413c6811ff9 (DEFAULT_TABLE table=test-table [id=98a8eea3d4a74128a7759db251e75d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:57.834422 15456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0879bcc51add439583920413c6811ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:57.836583 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.811860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:39687 (local address 127.0.0.1:40114)
0809 12:04:57.824825 (+ 12965us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:57.824850 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:57.824983 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:57.829305 (+ 4322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:57.829310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:57.829334 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:57.829517 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.829529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.831468 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.831479 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.833666 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.833681 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.835643 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.835651 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.835878 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.835947 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 12:04:57.836057 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 12:04:57.836206 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12831,"mutex_wait_us":706,"thread_start_us":15690,"threads_started":1}
I20260809 12:04:57.836640 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.812242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40114 (local address 127.0.147.66:39687)
0809 12:04:57.828543 (+ 16301us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:57.828555 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:57.828601 (+ 46us) server_negotiation.cc:407] Connection header received
0809 12:04:57.828737 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:57.828749 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:57.829022 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:57.829305 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:57.829681 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.831299 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.833852 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.835356 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.835481 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.835699 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:57.836135 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 12:04:57.836305 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16087,"thread_start_us":16080,"threads_started":1}
I20260809 12:04:57.836735 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.813007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45987 (local address 127.0.0.1:54242)
0809 12:04:57.828421 (+ 15414us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:57.828439 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:57.828586 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:57.829557 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:57.829577 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:57.829607 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:57.829903 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.829910 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.831268 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.831276 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.833672 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.833681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.835885 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.835890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.836067 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.836101 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:04:57.836191 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 12:04:57.836320 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15288,"mutex_wait_us":83,"thread_start_us":15769,"threads_started":1}
I20260809 12:04:57.837366 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.813541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54242 (local address 127.0.147.67:45987)
0809 12:04:57.827712 (+ 14171us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:57.827725 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:57.828590 (+ 865us) server_negotiation.cc:407] Connection header received
0809 12:04:57.828815 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:57.828824 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:57.829069 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:57.829317 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:57.830096 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.831132 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.833861 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.835348 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.835485 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.835739 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:57.836854 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0809 12:04:57.837020 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13915,"thread_start_us":13765,"threads_started":1}
I20260809 12:04:57.841255 15876 tablet_service.cc:1467] Processing CreateTablet for tablet 0879bcc51add439583920413c6811ff9 (DEFAULT_TABLE table=test-table [id=98a8eea3d4a74128a7759db251e75d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:57.841833 15673 tablet_service.cc:1467] Processing CreateTablet for tablet 0879bcc51add439583920413c6811ff9 (DEFAULT_TABLE table=test-table [id=98a8eea3d4a74128a7759db251e75d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:04:57.842720 15876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0879bcc51add439583920413c6811ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:57.843158 15673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0879bcc51add439583920413c6811ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:04:57.847661 15998 tablet_bootstrap.cc:492] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: Bootstrap starting.
I20260809 12:04:57.851593 15998 tablet_bootstrap.cc:654] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:57.852629 15998 log.cc:824] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:57.855175 15998 tablet_bootstrap.cc:492] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: No bootstrap required, opened a new log
I20260809 12:04:57.855445 15998 ts_tablet_manager.cc:1397] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 12:04:57.857659 16002 tablet_bootstrap.cc:492] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: Bootstrap starting.
I20260809 12:04:57.858434 16001 tablet_bootstrap.cc:492] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: Bootstrap starting.
I20260809 12:04:57.861693 16002 tablet_bootstrap.cc:654] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:57.862607 16001 tablet_bootstrap.cc:654] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: Neither blocks nor log segments found. Creating new log.
I20260809 12:04:57.862663 16002 log.cc:824] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:57.863806 16001 log.cc:824] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:04:57.865551 16002 tablet_bootstrap.cc:492] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: No bootstrap required, opened a new log
I20260809 12:04:57.865476 15998 raft_consensus.cc:348] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } }
I20260809 12:04:57.865819 16002 ts_tablet_manager.cc:1397] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 12:04:57.865912 15998 raft_consensus.cc:374] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:57.866037 15998 raft_consensus.cc:729] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e88e9df892c848fe8c4bb68b8d9dc842, State: Initialized, Role: FOLLOWER
I20260809 12:04:57.866767 16001 tablet_bootstrap.cc:492] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: No bootstrap required, opened a new log
I20260809 12:04:57.866667 15998 consensus_queue.cc:260] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } }
I20260809 12:04:57.867024 16001 ts_tablet_manager.cc:1397] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 12:04:57.872893 15998 ts_tablet_manager.cc:1428] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: Time spent starting tablet: real 0.017s user 0.011s sys 0.007s
I20260809 12:04:57.873576 16002 raft_consensus.cc:348] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } }
I20260809 12:04:57.873991 16002 raft_consensus.cc:374] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:57.874102 16002 raft_consensus.cc:729] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ba6a75019b64b178675142070345bad, State: Initialized, Role: FOLLOWER
I20260809 12:04:57.874814 16002 consensus_queue.cc:260] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } }
I20260809 12:04:57.875897 15967 heartbeater.cc:503] Master 127.0.147.126:45005 was elected leader, sending a full tablet report...
I20260809 12:04:57.876539 16002 ts_tablet_manager.cc:1428] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260809 12:04:57.879993 16001 raft_consensus.cc:348] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } }
I20260809 12:04:57.880445 16001 raft_consensus.cc:374] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:04:57.880584 16001 raft_consensus.cc:729] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de606321aafd4d17b31ce8439b261e0f, State: Initialized, Role: FOLLOWER
I20260809 12:04:57.881088 16001 consensus_queue.cc:260] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } }
I20260809 12:04:57.882130 15777 heartbeater.cc:503] Master 127.0.147.126:45005 was elected leader, sending a full tablet report...
I20260809 12:04:57.882552 16001 ts_tablet_manager.cc:1428] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260809 12:04:57.919169 15571 tablet.cc:2369] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:04:57.934190 16012 raft_consensus.cc:482] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:04:57.934489 16012 raft_consensus.cc:504] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } }
I20260809 12:04:57.936333 16012 leader_election.cc:288] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e88e9df892c848fe8c4bb68b8d9dc842 (127.0.147.65:33757), de606321aafd4d17b31ce8439b261e0f (127.0.147.66:39687)
I20260809 12:04:57.942957 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.936497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:33757 (local address 127.0.147.67:46569)
0809 12:04:57.936710 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:57.936739 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:57.936874 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:57.937408 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:57.937430 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:57.937473 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:57.937716 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.937730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.939108 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.939112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.941081 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.941096 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.942432 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.942436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.942573 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.942597 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:04:57.942669 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 12:04:57.942755 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":20}
I20260809 12:04:57.943116 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.936736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:46569 (local address 127.0.147.65:33757)
0809 12:04:57.936953 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:57.936959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:57.936981 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:04:57.937084 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:57.937093 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:57.937188 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:57.937314 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:57.937886 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.938990 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.941261 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.942293 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.942447 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.942521 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:57.942726 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 12:04:57.942855 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:04:57.944135 16014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.936841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:39687 (local address 127.0.147.67:43991)
0809 12:04:57.937713 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 12:04:57.937730 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:04:57.937856 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:04:57.938204 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:04:57.938208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:04:57.938271 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:04:57.938558 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.938566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.939772 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.939778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.941730 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:04:57.941742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.943368 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.943375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:04:57.943615 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.943651 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:04:57.943745 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 12:04:57.943867 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":21,"thread_start_us":483,"threads_started":1}
I20260809 12:04:57.944185 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.937063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:43991 (local address 127.0.147.66:39687)
0809 12:04:57.937290 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:57.937297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:57.937835 (+ 538us) server_negotiation.cc:407] Connection header received
0809 12:04:57.937985 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:57.937993 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:57.938078 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:57.938166 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:04:57.938701 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.939617 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.941886 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.943079 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.943242 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.943325 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:57.943784 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 12:04:57.943919 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:04:57.944131 15495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0879bcc51add439583920413c6811ff9" candidate_uuid: "9ba6a75019b64b178675142070345bad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" is_pre_election: true
I20260809 12:04:57.944758 15495 raft_consensus.cc:2454] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ba6a75019b64b178675142070345bad in term 0.
I20260809 12:04:57.945079 15702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0879bcc51add439583920413c6811ff9" candidate_uuid: "9ba6a75019b64b178675142070345bad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de606321aafd4d17b31ce8439b261e0f" is_pre_election: true
I20260809 12:04:57.945448 15812 leader_election.cc:302] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ba6a75019b64b178675142070345bad, e88e9df892c848fe8c4bb68b8d9dc842; no voters:
I20260809 12:04:57.945636 15702 raft_consensus.cc:2454] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ba6a75019b64b178675142070345bad in term 0.
I20260809 12:04:57.945895 16012 raft_consensus.cc:2781] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:04:57.946063 16012 raft_consensus.cc:482] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:04:57.946197 16012 raft_consensus.cc:3037] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 0 FOLLOWER]: Advancing to term 1
W20260809 12:04:57.947166 15778 tablet.cc:2369] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:04:57.948626 16012 raft_consensus.cc:504] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } }
I20260809 12:04:57.949266 16012 leader_election.cc:288] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [CANDIDATE]: Term 1 election: Requested vote from peers e88e9df892c848fe8c4bb68b8d9dc842 (127.0.147.65:33757), de606321aafd4d17b31ce8439b261e0f (127.0.147.66:39687)
I20260809 12:04:57.949669 15495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0879bcc51add439583920413c6811ff9" candidate_uuid: "9ba6a75019b64b178675142070345bad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e88e9df892c848fe8c4bb68b8d9dc842"
I20260809 12:04:57.949738 15702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0879bcc51add439583920413c6811ff9" candidate_uuid: "9ba6a75019b64b178675142070345bad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de606321aafd4d17b31ce8439b261e0f"
I20260809 12:04:57.949851 15495 raft_consensus.cc:3037] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:57.949988 15702 raft_consensus.cc:3037] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:04:57.952158 15495 raft_consensus.cc:2454] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ba6a75019b64b178675142070345bad in term 1.
I20260809 12:04:57.952296 15702 raft_consensus.cc:2454] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ba6a75019b64b178675142070345bad in term 1.
I20260809 12:04:57.952564 15812 leader_election.cc:302] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ba6a75019b64b178675142070345bad, e88e9df892c848fe8c4bb68b8d9dc842; no voters:
I20260809 12:04:57.952838 16012 raft_consensus.cc:2781] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:04:57.953285 16012 raft_consensus.cc:686] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 1 LEADER]: Becoming Leader. State: Replica: 9ba6a75019b64b178675142070345bad, State: Running, Role: LEADER
I20260809 12:04:57.953740 16012 consensus_queue.cc:237] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } }
I20260809 12:04:57.958431 15272 catalog_manager.cc:5495] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ba6a75019b64b178675142070345bad (127.0.147.67). New cstate: current_term: 1 leader_uuid: "9ba6a75019b64b178675142070345bad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ba6a75019b64b178675142070345bad" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 } health_report { overall_health: UNKNOWN } } }
I20260809 12:04:57.973435 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:57.968611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54244 (local address 127.0.147.67:45987)
0809 12:04:57.968782 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:57.968787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:57.968805 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:04:57.968912 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:57.968915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:57.968987 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:57.969105 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:04:57.969434 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.970160 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.971398 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:57.971854 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:57.971947 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:57.972383 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:04:57.972908 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:04:57.973021 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:57.973098 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 12:04:57.973159 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 12:04:57.981671 15495 raft_consensus.cc:1261] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [term 1 FOLLOWER]: Refusing update from remote peer 9ba6a75019b64b178675142070345bad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:57.981673 15702 raft_consensus.cc:1261] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [term 1 FOLLOWER]: Refusing update from remote peer 9ba6a75019b64b178675142070345bad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:04:57.982656 16016 consensus_queue.cc:1035] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [LEADER]: Connected to new peer: Peer: permanent_uuid: "de606321aafd4d17b31ce8439b261e0f" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:04:57.982988 16012 consensus_queue.cc:1035] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e88e9df892c848fe8c4bb68b8d9dc842" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:04:57.989727 15968 tablet.cc:2369] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:04:57.994127 16021 mvcc.cc:205] Tried to move back new op lower bound from 7316590993322868736 to 7316590993222164480. Current Snapshot: MvccSnapshot[applied={T|T < 7316590993322868736}]
I20260809 12:04:57.994740 16024 mvcc.cc:205] Tried to move back new op lower bound from 7316590993322868736 to 7316590993222164480. Current Snapshot: MvccSnapshot[applied={T|T < 7316590993322868736}]
I20260809 12:04:58.010303 15272 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:58.010596 15272 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:04:58.014279 15272 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e1921d401a9498c97c24e67f5c8b1bf: Sending DeleteTablet for 3 replicas of tablet 0879bcc51add439583920413c6811ff9
I20260809 12:04:58.015426 15456 tablet_service.cc:1514] Processing DeleteTablet for tablet 0879bcc51add439583920413c6811ff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:58 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:45396
I20260809 12:04:58.015519 15876 tablet_service.cc:1514] Processing DeleteTablet for tablet 0879bcc51add439583920413c6811ff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:58 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:54242
I20260809 12:04:58.015905 15673 tablet_service.cc:1514] Processing DeleteTablet for tablet 0879bcc51add439583920413c6811ff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:04:58 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:40114
I20260809 12:04:58.018394 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 15368
I20260809 12:04:58.030483 16035 tablet_replica.cc:330] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: stopping tablet replica
I20260809 12:04:58.031081 16034 tablet_replica.cc:330] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: stopping tablet replica
I20260809 12:04:58.031713 16034 raft_consensus.cc:2229] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:58.032277 16035 raft_consensus.cc:2229] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:04:58.032889 16034 raft_consensus.cc:2258] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:58.033593 16035 raft_consensus.cc:2258] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:58.034791 16042 tablet_replica.cc:330] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: stopping tablet replica
I20260809 12:04:58.035280 16042 raft_consensus.cc:2229] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:04:58.035857 16042 raft_consensus.cc:2258] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:04:58.037060 16035 ts_tablet_manager.cc:1905] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:58.037181 16034 ts_tablet_manager.cc:1905] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 12:04:58 dist-test-slave-sb2z krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:04:58.044983 16034 ts_tablet_manager.cc:1918] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:58.045172 16034 log.cc:1196] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/0879bcc51add439583920413c6811ff9
I20260809 12:04:58.045760 16034 ts_tablet_manager.cc:1939] T 0879bcc51add439583920413c6811ff9 P e88e9df892c848fe8c4bb68b8d9dc842: Deleting consensus metadata
I20260809 12:04:58.046772 16042 ts_tablet_manager.cc:1905] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:04:58.047698 15253 catalog_manager.cc:4841] TS e88e9df892c848fe8c4bb68b8d9dc842 (127.0.147.65:33757): tablet 0879bcc51add439583920413c6811ff9 (table test-table [id=98a8eea3d4a74128a7759db251e75d9c]) successfully deleted
I20260809 12:04:58.048669 16035 ts_tablet_manager.cc:1918] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:58.048861 16035 log.cc:1196] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/0879bcc51add439583920413c6811ff9
I20260809 12:04:58.049501 16035 ts_tablet_manager.cc:1939] T 0879bcc51add439583920413c6811ff9 P 9ba6a75019b64b178675142070345bad: Deleting consensus metadata
I20260809 12:04:58.051942 15253 catalog_manager.cc:4841] TS 9ba6a75019b64b178675142070345bad (127.0.147.67:45987): tablet 0879bcc51add439583920413c6811ff9 (table test-table [id=98a8eea3d4a74128a7759db251e75d9c]) successfully deleted
I20260809 12:04:58.054407 16042 ts_tablet_manager.cc:1918] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:04:58.054622 16042 log.cc:1196] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/0879bcc51add439583920413c6811ff9
I20260809 12:04:58.055235 16042 ts_tablet_manager.cc:1939] T 0879bcc51add439583920413c6811ff9 P de606321aafd4d17b31ce8439b261e0f: Deleting consensus metadata
I20260809 12:04:58.056530 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:58.020318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.0.147.65:33757)
0809 12:04:58.020552 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:58.020561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:58.024352 (+ 3791us) server_negotiation.cc:407] Connection header received
0809 12:04:58.025551 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:58.025560 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:58.025696 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:58.025795 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:58.029963 (+ 4168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:58.031145 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:58.032998 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:58.033667 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:58.033753 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:58.049244 (+ 15491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:58.049260 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:58.049277 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:58.049332 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:58.052958 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:58.053873 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:58.053882 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:58.053886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:58.054062 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:58.054552 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:58.054557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:58.054560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:58.055008 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:58.055297 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:58.055909 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 12:04:58.056138 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:04:58.057431 15254 catalog_manager.cc:4841] TS de606321aafd4d17b31ce8439b261e0f (127.0.147.66:39687): tablet 0879bcc51add439583920413c6811ff9 (table test-table [id=98a8eea3d4a74128a7759db251e75d9c]) successfully deleted
I20260809 12:04:58.057992 15559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:58.211712 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 15368
I20260809 12:04:58.237126 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 15577
Aug 09 12:04:58 dist-test-slave-sb2z krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:04:58.253221 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:58.238011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.0.147.66:39687)
0809 12:04:58.238222 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:58.238227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:58.238249 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:04:58.238456 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:58.238462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:58.238551 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:58.238673 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:58.240129 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:58.240922 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:58.242432 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:58.242827 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:58.242983 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:58.247324 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:58.247339 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:58.247354 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:58.247385 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:58.250268 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:58.250932 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:58.250940 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:58.250948 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:58.251069 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:58.251451 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:58.251460 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:58.251463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:58.252024 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:58.252324 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:58.252653 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 12:04:58.252887 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 12:04:58.253964 15768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:58.345129 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 15577
I20260809 12:04:58.371178 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 15784
Aug 09 12:04:58 dist-test-slave-sb2z krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:04:58.389066 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:04:58.372299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.0.147.67:45987)
0809 12:04:58.372514 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:04:58.372521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:04:58.372545 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:04:58.372717 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:04:58.372723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:04:58.372845 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:04:58.372978 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:04:58.374312 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:58.375540 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:58.376951 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:04:58.377700 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:04:58.377857 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:04:58.382817 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:04:58.382830 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:04:58.382846 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:04:58.382884 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:04:58.385968 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:58.386834 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:58.386842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:58.386844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:58.387011 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:04:58.387468 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:04:58.387471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:04:58.387473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:04:58.387876 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:04:58.388112 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 12:04:58.388494 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 12:04:58.388671 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:04:58.389779 15960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:58.517647 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 15784
I20260809 12:04:58.545154 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 15227
I20260809 12:04:58.546383 15318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:04:58.657788 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 15227
2026-08-09T12:04:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4702 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:04:58 dist-test-slave-sb2z krb5kdc[16054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:04:58 dist-test-slave-sb2z krb5kdc[16054](info): set up 2 sockets
Aug 09 12:04:58 dist-test-slave-sb2z krb5kdc[16054](info): commencing operation
krb5kdc: starting...
W20260809 12:04:59.969965 589 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:05:00 dist-test-slave-sb2z krb5kdc[16054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:05:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:05:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.147.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.147.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/rangeradmin_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/rangeradmin_127.0.147.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.147.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.147.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/rangerlookup_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/rangerlookup_127.0.147.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.147.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.147.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/HTTP_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/HTTP_127.0.147.84@KRBTEST.COM.keytab.
I20260809 12:05:00.394645 589 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task3WUkck/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/postgres -l logfile start
2026-08-09 12:05:02.866 UTC [16105] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 12:05:02.866 UTC [16105] LOG: listening on IPv4 address "127.0.147.84", port 44229
2026-08-09 12:05:02.869 UTC [16105] LOG: listening on Unix socket "/tmp/.s.PGSQL.44229"
2026-08-09 12:05:02.895 UTC [16109] LOG: database system was shut down at 2026-08-09 12:05:01 UTC
2026-08-09 12:05:02.898 UTC [16105] LOG: database system is ready to accept connections
I20260809 12:05:04.060879 589 mini_postgres.cc:96] Postgres bound to 44229
2026-08-09 12:05:04.090 UTC [16120] FATAL: database "slave" does not exist
127.0.147.84:44229 - accepting connections
I20260809 12:05:04.091776 589 mini_ranger.cc:161] Starting Ranger...
I20260809 12:05:04.126652 589 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 12:05:04.593479 589 mini_ranger.cc:87] Created ranger Postgres database
I20260809 12:05:04.593634 589 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-admin
2026-08-09 12:05:04,958 [I] DB FLAVOR :POSTGRES
2026-08-09 12:05:04,959 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:05:04,959 [I] Checking connection..
2026-08-09 12:05:04,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 12:05:05,247 [I] Checking connection passed.
2026-08-09 12:05:05,247 [I] --------- Verifying version history table ---------
2026-08-09 12:05:05,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:05:05,487 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:05:05,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:05:05,755 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:05:05,755 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 12:05:05,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 12:05:06.025 UTC [16218] WARNING: there is no transaction in progress
2026-08-09 12:05:06,039 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 12:05:06,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:05:06,297 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 12:05:06,298 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:05:06,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 12:05:06,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 12:05:06,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-09 12:05:07,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:05:07,337 [I] Table x_portal_user does not exist in database ranger
2026-08-09 12:05:07,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:05:07,601 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 12:05:07,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:05:07,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:05:08,131 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 12:05:08,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 12:05:09.094 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.110 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.125 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.140 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.155 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.172 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.235 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.242 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.255 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.265 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.275 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.287 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.296 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.303 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.309 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09.316 UTC [16368] WARNING: there is no transaction in progress
2026-08-09 12:05:09,928 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 12:05:09,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 12:05:10,177 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 12:05:10,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:05:10,413 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 12:05:10,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:05:10,662 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 12:05:10,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:05:10,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:05:11,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-09 12:05:11,435 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 12:05:11,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 12:05:11,698 [I] DB_PATCHES have already been applied
I20260809 12:05:11.704838 589 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-admin:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task3WUkck/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 12:05:11.704936 589 mini_ranger.cc:193] Using host: 127.0.147.84
I20260809 12:05:11.730103 589 mini_ranger.cc:227] Ranger admin URL: http://127.0.147.84:58271
Aug 09, 2026 12:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 12:05:13.069790 589 mini_ranger.cc:160] Time spent starting Ranger: real 8.978s user 0.004s sys 0.070s
Aug 09, 2026 12:05:32 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:05:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 09, 2026 12:05:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:05:32 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:05:33 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 12:05:34.399587 589 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.147.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.147.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/rangerkms_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/rangerkms_127.0.147.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.147.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.147.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.147.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/HTTP_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.147.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/HTTP_127.0.147.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 12:05:34 dist-test-slave-sb2z krb5kdc[16054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277134, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 12:05:34.619740 589 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 12:05:34.649121 589 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 12:05:35.415710 589 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 12:05:35.415921 589 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-kms
I20260809 12:05:35.416074 589 mini_ranger_kms.cc:227] Using postgres at 127.0.147.84:44229
2026-08-09 12:05:35,981 [I] DB FLAVOR :POSTGRES
2026-08-09 12:05:35,981 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:05:35,982 [I] Checking connection
2026-08-09 12:05:35,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 12:05:36,225 [I] connection success
2026-08-09 12:05:36,226 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 12:05:36,226 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 12:05:36,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 12:05:36,512 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 12:05:36,513 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:05:36,513 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 12:05:36,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:44229/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 12:05:36,775 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 12:05:37.243458 589 mini_ranger_kms.cc:310] Created kms service
I20260809 12:05:37.446309 589 mini_ranger_kms.cc:326] Created kudu user
I20260809 12:05:37.493979 589 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 12:05:37 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277134, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:05:37.906536 589 mini_ranger_kms.cc:384] Added ranger policy
I20260809 12:05:37.907958 589 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-kms:/tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 12:05:37.908036 589 mini_ranger_kms.cc:240] Using host: 127.0.147.84
I20260809 12:05:37.938015 589 mini_ranger_kms.cc:276] Ranger KMS PID: 16628
I20260809 12:05:37.938215 589 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.147.84:57801
Aug 09, 2026 12:05:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:05:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:05:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:05:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:05:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:05:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 12:05:40 dist-test-slave-sb2z krb5kdc[16054](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277140, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.147.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 12:05:40 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277140, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.147.84@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:05:40.896975 589 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.277s user 0.148s sys 0.414s
I20260809 12:05:40.897372 589 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 12:05:40.897629 589 mini_ranger_kms.cc:401] 127.0.147.84:57801/kms/v1/keys
Aug 09 12:05:41 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277140, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.147.84@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
Aug 09 12:05:43 dist-test-slave-sb2z krb5kdc[16054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:05:43.925872 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:46201
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:46201
--ranger_config_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:57801/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:05:44.485028 16707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:05:44.485488 16707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:05:44.485661 16707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:05:44.494374 16707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:05:44.494575 16707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:05:44.494700 16707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:05:44.494834 16707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:05:44.494971 16707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:05:44.507519 16707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.147.84:57801/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:46201
--ranger_config_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:46201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:05:44.509105 16707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:05:44.510639 16707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:05:44.528970 16714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:05:44.528918 16715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:05:44.532070 16718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:05:44.535446 16707 server_base.cc:1034] running on GCE node
I20260809 12:05:44.536744 16707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:05:44.538404 16707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:05:44.539649 16707 hybrid_clock.cc:648] HybridClock initialized: now 1786277144539581 us; error 50 us; skew 500 ppm
I20260809 12:05:44.540047 16707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:05:44 dist-test-slave-sb2z krb5kdc[16054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277144, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:05:44.548015 16707 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:05:44.550429 16707 webserver.cc:460] Webserver started at http://127.0.147.126:37447/ using document root <none> and password file <none>
I20260809 12:05:44.551229 16707 fs_manager.cc:359] Metadata directory not provided
I20260809 12:05:44.551473 16707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:05:44.551887 16707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:05:44 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277144, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:05:44.622123 16707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4"
format_stamp: "Formatted at 2026-08-09 12:05:44 on dist-test-slave-sb2z"
server_key: "48cb4c8d4b7b23506bc633bc440912e8"
server_key_iv: "131ef1609926d26e528a61ad86bd4384"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:44.623287 16707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4"
format_stamp: "Formatted at 2026-08-09 12:05:44 on dist-test-slave-sb2z"
server_key: "48cb4c8d4b7b23506bc633bc440912e8"
server_key_iv: "131ef1609926d26e528a61ad86bd4384"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:44.630206 16707 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 12:05:44.650744 16725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:05:44.652578 16707 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260809 12:05:44.652837 16707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4"
format_stamp: "Formatted at 2026-08-09 12:05:44 on dist-test-slave-sb2z"
server_key: "48cb4c8d4b7b23506bc633bc440912e8"
server_key_iv: "131ef1609926d26e528a61ad86bd4384"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:44.653123 16707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:05:44.684244 16707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:05:44.688874 16707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:05:44.689222 16707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:05:44.718295 16707 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:46201
I20260809 12:05:44.718353 16807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:46201 every 8 connection(s)
I20260809 12:05:44.719874 16707 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:05:44.724125 16808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:05:44.728464 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 16707
I20260809 12:05:44.728711 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:05:44.729399 589 external_mini_cluster.cc:1428] Setting key 62e166a76151097a41ec19966e2338c2
I20260809 12:05:44.738054 16808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4: Bootstrap starting.
I20260809 12:05:44.742522 16808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4: Neither blocks nor log segments found. Creating new log.
I20260809 12:05:44.744235 16808 log.cc:824] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:05:44 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:05:44.747716 16808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4: No bootstrap required, opened a new log
I20260809 12:05:44.758424 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:44.731545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.0.147.126:46201)
0809 12:05:44.732611 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:44.732627 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:44.732677 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:05:44.736279 (+ 3602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:44.736310 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:44.736833 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:44.737395 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:05:44.738583 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:44.741048 (+ 2465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:44.742367 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:44.743626 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:44.743805 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:44.748398 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:05:44.748410 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:05:44.748430 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:05:44.748476 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:05:44.753408 (+ 4932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:44.754098 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:44.754107 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:44.754115 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:44.754353 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:44.754794 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:44.754799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:44.754805 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:44.755622 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:05:44.755987 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:44.756346 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 12:05:44.756805 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":511,"threads_started":1}
I20260809 12:05:44.760941 16808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46201 } }
I20260809 12:05:44.761514 16808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:05:44.761668 16808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ecd58cd2e24d1cb950ddd1e2ba15f4, State: Initialized, Role: FOLLOWER
I20260809 12:05:44.762387 16808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46201 } }
I20260809 12:05:44.762686 16808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:05:44.762856 16808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:05:44.763077 16808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:05:44.766139 16808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46201 } }
I20260809 12:05:44.766666 16808 leader_election.cc:302] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03ecd58cd2e24d1cb950ddd1e2ba15f4; no voters:
I20260809 12:05:44.767386 16808 leader_election.cc:288] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:05:44.767614 16814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:05:44.768703 16814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [term 1 LEADER]: Becoming Leader. State: Replica: 03ecd58cd2e24d1cb950ddd1e2ba15f4, State: Running, Role: LEADER
I20260809 12:05:44.769325 16814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46201 } }
I20260809 12:05:44.770167 16808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:05:44.772607 16815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46201 } } }
I20260809 12:05:44.772641 16816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03ecd58cd2e24d1cb950ddd1e2ba15f4. Latest consensus state: current_term: 1 leader_uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ecd58cd2e24d1cb950ddd1e2ba15f4" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46201 } } }
I20260809 12:05:44.773303 16815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:05:44.773348 16816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:05:44.777397 16820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:05:44.784142 16820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:05:44.786877 16808 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 12:05:44.809059 16820 catalog_manager.cc:1347] Generated new cluster ID: 4e2874fc2e1c4815b16b1e1e7215a365
I20260809 12:05:44.809508 16820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:05:44.838443 16820 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:05:44.840179 16820 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:05:44.849572 16820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4: Generated new TSK 0
I20260809 12:05:44.850484 16820 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:05:45.727250 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:57801/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46201
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:05:46 dist-test-slave-sb2z krb5kdc[16054](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 12:05:46.356175 16871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:05:46.356736 16871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:05:46.356988 16871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:05:46.372339 16871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:05:46.372632 16871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:05:46.372969 16871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:05:46.394845 16871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.147.84:57801/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:05:46.397097 16871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:05:46.399461 16871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:05:46.420147 16890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:05:46.422717 16892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:05:46.430100 16871 server_base.cc:1034] running on GCE node
W20260809 12:05:46.430651 16895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:05:46.431561 16871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:05:46.432921 16871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:05:46.434232 16871 hybrid_clock.cc:648] HybridClock initialized: now 1786277146434126 us; error 82 us; skew 500 ppm
I20260809 12:05:46.434743 16871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:05:46 dist-test-slave-sb2z krb5kdc[16054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:05:46.440071 16871 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:05:46.442874 16871 webserver.cc:460] Webserver started at http://127.0.147.65:43751/ using document root <none> and password file <none>
I20260809 12:05:46.443848 16871 fs_manager.cc:359] Metadata directory not provided
I20260809 12:05:46.444036 16871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:05:46.444458 16871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:05:46 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:05:46.468164 16871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "caa6404eb0b645f29565570c2ec12a98"
format_stamp: "Formatted at 2026-08-09 12:05:46 on dist-test-slave-sb2z"
server_key: "c854001da57d5ac9a7999009b86447b4"
server_key_iv: "50ac5dc21f9397de734fb96e9634ca12"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:46.470085 16871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "caa6404eb0b645f29565570c2ec12a98"
format_stamp: "Formatted at 2026-08-09 12:05:46 on dist-test-slave-sb2z"
server_key: "c854001da57d5ac9a7999009b86447b4"
server_key_iv: "50ac5dc21f9397de734fb96e9634ca12"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:46.481913 16871 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.009s sys 0.000s
I20260809 12:05:46.526543 16908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:05:46.529326 16871 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.008s sys 0.004s
I20260809 12:05:46.529546 16871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "caa6404eb0b645f29565570c2ec12a98"
format_stamp: "Formatted at 2026-08-09 12:05:46 on dist-test-slave-sb2z"
server_key: "c854001da57d5ac9a7999009b86447b4"
server_key_iv: "50ac5dc21f9397de734fb96e9634ca12"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:46.529844 16871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:05:46.583099 16871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:05:46.591558 16871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:05:46.592185 16871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:05:46.604326 16871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:05:46.620942 16871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:05:46.621170 16871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.001s
I20260809 12:05:46.621572 16871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:05:46.621812 16871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:05:46 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:05:46.828023 16871 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:46125
I20260809 12:05:46.830209 16871 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:05:46.828001 17070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:46125 every 8 connection(s)
I20260809 12:05:46.835024 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 16871
I20260809 12:05:46.835224 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:05:46.836086 589 external_mini_cluster.cc:1428] Setting key e27e2a378f5770e38db3ba23924e6d9e
Aug 09 12:05:46 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:05:46.868525 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:46.834259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:45409 (local address 127.0.147.126:46201)
0809 12:05:46.835169 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:46.835177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:46.836654 (+ 1477us) server_negotiation.cc:407] Connection header received
0809 12:05:46.841616 (+ 4962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:46.841622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:46.841704 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:46.841918 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:05:46.844945 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:46.846644 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:46.849712 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:46.850678 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:46.850811 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:46.857576 (+ 6765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:05:46.857594 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:05:46.857600 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:05:46.857665 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:05:46.863363 (+ 5698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:46.864634 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:46.864639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:46.864643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:46.864916 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:46.865775 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:46.865783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:46.865788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:46.866281 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:05:46.866432 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:46.867724 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0809 12:05:46.867994 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":2686,"threads_started":1}
I20260809 12:05:46.870265 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:46.834802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:46201 (local address 127.0.147.65:45409)
0809 12:05:46.836319 (+ 1517us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:46.836367 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:05:46.841259 (+ 4892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:05:46.842618 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:05:46.842633 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:05:46.843617 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:05:46.844725 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:46.844754 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:46.846935 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:46.846941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:46.849520 (+ 2579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:46.849556 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:46.851325 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:46.851336 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:46.851825 (+ 489us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:46.852963 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:05:46.852998 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:05:46.857190 (+ 4192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:05:46.863648 (+ 6458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:05:46.863657 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:05:46.863676 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:05:46.864453 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:05:46.865120 (+ 667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:05:46.865125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:05:46.865129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:05:46.865574 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:05:46.866575 (+ 1001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:05:46.866583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:05:46.867138 (+ 555us) client_negotiation.cc:769] Sending connection context
0809 12:05:46.867554 (+ 416us) client_negotiation.cc:240] Negotiation successful
0809 12:05:46.868312 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1176,"thread_start_us":826,"threads_started":1}
I20260809 12:05:46.872288 17071 heartbeater.cc:347] Connected to a master server at 127.0.147.126:46201
I20260809 12:05:46.872681 17071 heartbeater.cc:464] Registering TS with master...
I20260809 12:05:46.873741 17071 heartbeater.cc:511] Master 127.0.147.126:46201 requested a full tablet report, sending...
I20260809 12:05:46.876791 16753 ts_manager.cc:194] Registered new tserver with Master: caa6404eb0b645f29565570c2ec12a98 (127.0.147.65:46125)
I20260809 12:05:46.879215 16753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:45409
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:05:46.931095 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:57801/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46201
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:05:47 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
W20260809 12:05:47.694721 17081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:05:47.695174 17081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:05:47.695437 17081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:05:47.705858 17081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:05:47.706045 17081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:05:47.706233 17081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:05:47.718178 17081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.147.84:57801/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:05:47.720791 17081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:05:47.722688 17081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:05:47.750284 17102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:05:47.752591 17103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:05:47.758072 17105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:05:47.762053 17081 server_base.cc:1034] running on GCE node
I20260809 12:05:47.762931 17081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:05:47.767984 17081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:05:47.769613 17081 hybrid_clock.cc:648] HybridClock initialized: now 1786277147769504 us; error 53 us; skew 500 ppm
I20260809 12:05:47.770164 17081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:05:47 dist-test-slave-sb2z krb5kdc[16054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:05:47.776933 17081 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:05:47.781937 17081 webserver.cc:460] Webserver started at http://127.0.147.66:39883/ using document root <none> and password file <none>
I20260809 12:05:47.783082 17081 fs_manager.cc:359] Metadata directory not provided
I20260809 12:05:47.783229 17081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:05:47.783656 17081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:05:47 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:05:47.806036 17081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "8819bbedc77b492ba739c36f788fc0a0"
format_stamp: "Formatted at 2026-08-09 12:05:47 on dist-test-slave-sb2z"
server_key: "a341d691784837de5d8257ec88bf0ae9"
server_key_iv: "a27ef641496f618a66da81b97c37ca7c"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:47.807339 17081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "8819bbedc77b492ba739c36f788fc0a0"
format_stamp: "Formatted at 2026-08-09 12:05:47 on dist-test-slave-sb2z"
server_key: "a341d691784837de5d8257ec88bf0ae9"
server_key_iv: "a27ef641496f618a66da81b97c37ca7c"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:47.824873 17081 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.005s sys 0.010s
I20260809 12:05:47.868939 17112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:05:47.871132 17081 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.004s sys 0.016s
I20260809 12:05:47.871388 17081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "8819bbedc77b492ba739c36f788fc0a0"
format_stamp: "Formatted at 2026-08-09 12:05:47 on dist-test-slave-sb2z"
server_key: "a341d691784837de5d8257ec88bf0ae9"
server_key_iv: "a27ef641496f618a66da81b97c37ca7c"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:47.871798 17081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:05:47.883278 17071 heartbeater.cc:503] Master 127.0.147.126:46201 was elected leader, sending a full tablet report...
I20260809 12:05:47.942253 17081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:05:47.950608 17081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:05:47.951339 17081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:05:47.953536 17081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:05:47.956593 17081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:05:47.956728 17081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:05:47.957019 17081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:05:47.957168 17081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:05:48.080487 17081 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:39867
I20260809 12:05:48.080511 17272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:39867 every 8 connection(s)
I20260809 12:05:48.082042 17081 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:05:48.084551 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 17081
I20260809 12:05:48.084751 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:05:48.085331 589 external_mini_cluster.cc:1428] Setting key 896bfcbb52621df477a87dc6a29520c3
Aug 09 12:05:48 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:05:48.119608 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:48.087729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:46465 (local address 127.0.147.126:46201)
0809 12:05:48.088449 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:48.088460 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:48.092757 (+ 4297us) server_negotiation.cc:407] Connection header received
0809 12:05:48.095065 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:48.095072 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:48.095167 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:48.095315 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:05:48.097008 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:48.097862 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:48.099780 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:48.100279 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:48.100433 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:48.108604 (+ 8171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:05:48.108618 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:05:48.108623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:05:48.108691 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:05:48.114510 (+ 5819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:48.115544 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:48.115549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:48.115552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:48.115714 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:48.116353 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:48.116362 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:48.116368 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:48.116936 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:05:48.117243 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:48.118472 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0809 12:05:48.118765 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":3859,"threads_started":1}
I20260809 12:05:48.120584 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:48.087624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:46201 (local address 127.0.147.66:46465)
0809 12:05:48.092335 (+ 4711us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:48.092387 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:05:48.094766 (+ 2379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:05:48.095672 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:05:48.095681 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:05:48.096164 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:05:48.096795 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:48.096815 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:48.098058 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:48.098063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:48.099606 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:48.099630 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:48.100554 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:48.100563 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:48.100819 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:48.101596 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:05:48.101621 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:05:48.108242 (+ 6621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:05:48.114686 (+ 6444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:05:48.114695 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:05:48.114713 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:05:48.115364 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:05:48.115843 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:05:48.115847 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:05:48.115850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:05:48.116194 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:05:48.117153 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:05:48.117398 (+ 245us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:05:48.117858 (+ 460us) client_negotiation.cc:769] Sending connection context
0809 12:05:48.118235 (+ 377us) client_negotiation.cc:240] Negotiation successful
0809 12:05:48.118837 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4385,"thread_start_us":4133,"threads_started":1}
I20260809 12:05:48.122965 17276 heartbeater.cc:347] Connected to a master server at 127.0.147.126:46201
I20260809 12:05:48.123395 17276 heartbeater.cc:464] Registering TS with master...
I20260809 12:05:48.124459 17276 heartbeater.cc:511] Master 127.0.147.126:46201 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
I20260809 12:05:48.126556 16753 ts_manager.cc:194] Registered new tserver with Master: 8819bbedc77b492ba739c36f788fc0a0 (127.0.147.66:39867)
I20260809 12:05:48.127724 16753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:46465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:05:48.174661 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:57801/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46201
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:05:48.749442 17287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:05:48.749773 17287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:05:48.750015 17287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:05:48.758984 17287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:05:48.759137 17287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:05:48.759325 17287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:05:48.771472 17287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.147.84:57801/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:46201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:05:48.772948 17287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:05:48.774546 17287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:05:48.792028 17292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:05:48.792644 17294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:05:48.793042 17297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:05:48.793551 17287 server_base.cc:1034] running on GCE node
I20260809 12:05:48.794232 17287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:05:48.795243 17287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:05:48.796494 17287 hybrid_clock.cc:648] HybridClock initialized: now 1786277148796420 us; error 51 us; skew 500 ppm
I20260809 12:05:48.796873 17287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:05:48 dist-test-slave-sb2z krb5kdc[16054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:05:48.800673 17287 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:05:48.802627 17287 webserver.cc:460] Webserver started at http://127.0.147.67:45217/ using document root <none> and password file <none>
I20260809 12:05:48.803296 17287 fs_manager.cc:359] Metadata directory not provided
I20260809 12:05:48.803422 17287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:05:48.803792 17287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:05:48 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:05:48.823082 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "65a18508bbc84d0fa89b7d902220d793"
format_stamp: "Formatted at 2026-08-09 12:05:48 on dist-test-slave-sb2z"
server_key: "2978e73a095e44a11883df5e58f61f89"
server_key_iv: "556de328ff827bf475960ec570a985aa"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:48.824335 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "65a18508bbc84d0fa89b7d902220d793"
format_stamp: "Formatted at 2026-08-09 12:05:48 on dist-test-slave-sb2z"
server_key: "2978e73a095e44a11883df5e58f61f89"
server_key_iv: "556de328ff827bf475960ec570a985aa"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:48.832175 17287 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260809 12:05:48.850570 17306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:05:48.852463 17287 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:05:48.852686 17287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "65a18508bbc84d0fa89b7d902220d793"
format_stamp: "Formatted at 2026-08-09 12:05:48 on dist-test-slave-sb2z"
server_key: "2978e73a095e44a11883df5e58f61f89"
server_key_iv: "556de328ff827bf475960ec570a985aa"
server_key_version: "kuduclusterkey@0"
I20260809 12:05:48.853047 17287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:05:48.881832 17287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:05:48.886773 17287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:05:48.887305 17287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:05:48.888720 17287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:05:48.890748 17287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:05:48.890889 17287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:05:48.891067 17287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:05:48.891163 17287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:05:48.966737 17287 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:43121
I20260809 12:05:48.966794 17471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:43121 every 8 connection(s)
I20260809 12:05:48.968402 17287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:05:48.975222 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 17287
I20260809 12:05:48.975458 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:05:48.976415 589 external_mini_cluster.cc:1428] Setting key 0352cd1023746e8b32a9f57472dc35a3
Aug 09 12:05:48 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:05:48.997414 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:48.971802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:56505 (local address 127.0.147.126:46201)
0809 12:05:48.972820 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:48.972827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:48.974196 (+ 1369us) server_negotiation.cc:407] Connection header received
0809 12:05:48.977712 (+ 3516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:48.977717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:48.977781 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:48.977998 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:05:48.980311 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:48.981582 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:48.984326 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:48.984807 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:48.984913 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:48.989391 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:05:48.989396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:05:48.989399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:05:48.989442 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:05:48.993963 (+ 4521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:48.994770 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:48.994776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:48.994782 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:48.994947 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:48.995466 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:48.995471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:48.995474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:48.995847 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:05:48.996008 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:48.996697 (+ 689us) server_negotiation.cc:299] Negotiation successful
0809 12:05:48.996914 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":565,"threads_started":1}
I20260809 12:05:48.998708 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:48.972754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:46201 (local address 127.0.147.67:56505)
0809 12:05:48.973944 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:48.973992 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:05:48.977437 (+ 3445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:05:48.978276 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:05:48.978285 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:05:48.978937 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:05:48.980152 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:48.980174 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:48.981768 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:48.981772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:48.984121 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:48.984134 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:48.985082 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:48.985088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:48.985361 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:48.986394 (+ 1033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:05:48.986440 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:05:48.989163 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:05:48.994114 (+ 4951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:05:48.994130 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:05:48.994142 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:05:48.994621 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:05:48.995070 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:05:48.995075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:05:48.995080 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:05:48.995335 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:05:48.995982 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:05:48.995990 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:05:48.996319 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 12:05:48.996623 (+ 304us) client_negotiation.cc:240] Negotiation successful
0809 12:05:48.997220 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"thread_start_us":667,"threads_started":1}
I20260809 12:05:49.000214 17472 heartbeater.cc:347] Connected to a master server at 127.0.147.126:46201
I20260809 12:05:49.000489 17472 heartbeater.cc:464] Registering TS with master...
I20260809 12:05:49.001411 17472 heartbeater.cc:511] Master 127.0.147.126:46201 requested a full tablet report, sending...
I20260809 12:05:49.003010 16753 ts_manager.cc:194] Registered new tserver with Master: 65a18508bbc84d0fa89b7d902220d793 (127.0.147.67:43121)
I20260809 12:05:49.004334 16753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:56505
I20260809 12:05:49.005744 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:05:49.027794 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.011290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39506 (local address 127.0.147.126:46201)
0809 12:05:49.011463 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.011467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.012557 (+ 1090us) server_negotiation.cc:407] Connection header received
0809 12:05:49.012692 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.012696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.012841 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.012949 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:05:49.014749 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.015986 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.017552 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.018199 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.018294 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.021179 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:05:49.021183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:05:49.021186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:05:49.021217 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:05:49.024881 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:49.025604 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:49.025607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:49.025610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:49.025791 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:49.026169 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:49.026172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:49.026175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:49.026516 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:05:49.026841 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.027023 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.027312 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:05:49.032565 16753 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:05:49.042115 16753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:05:49.078739 17494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.061454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57248 (local address 127.0.147.65:46125)
0809 12:05:49.064128 (+ 2674us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.064140 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.064178 (+ 38us) server_negotiation.cc:407] Connection header received
0809 12:05:49.064301 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.064313 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.064643 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.065120 (+ 477us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:05:49.068413 (+ 3293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.070190 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.074983 (+ 4793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.077062 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.077238 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.077614 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.077943 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.078144 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2413,"thread_start_us":2142,"threads_started":1}
I20260809 12:05:49.079043 17491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.059219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43121 (local address 127.0.0.1:47220)
0809 12:05:49.060939 (+ 1720us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.060983 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:05:49.061342 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:05:49.067876 (+ 6534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:05:49.067880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:05:49.067917 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:05:49.068853 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.068865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.071292 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.071300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.075146 (+ 3846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.075170 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.077950 (+ 2780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.077961 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.078203 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.078244 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 12:05:49.078334 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 12:05:49.078570 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1219,"thread_start_us":898,"threads_started":1}
I20260809 12:05:49.079943 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.060489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46125 (local address 127.0.0.1:57248)
0809 12:05:49.062082 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.062100 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:05:49.062411 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:05:49.065175 (+ 2764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:05:49.065187 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:05:49.065231 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:05:49.067574 (+ 2343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.067596 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.070373 (+ 2777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.070377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.074737 (+ 4360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.074752 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.077340 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.077349 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.077653 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.077720 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 12:05:49.079043 (+ 1323us) client_negotiation.cc:240] Negotiation successful
0809 12:05:49.079212 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1457,"thread_start_us":1303,"threads_started":1}
I20260809 12:05:49.081554 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.062291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:39867 (local address 127.0.0.1:55034)
0809 12:05:49.069086 (+ 6795us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.069103 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:05:49.069370 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:05:49.070161 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:05:49.070166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:05:49.070191 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:05:49.072235 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.072248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.073968 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.073973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.076675 (+ 2702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.076692 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.080311 (+ 3619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.080322 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.080556 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.080597 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 12:05:49.080726 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 12:05:49.080880 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5768,"thread_start_us":5413,"threads_started":1}
I20260809 12:05:49.081544 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.062600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.0.147.66:39867)
0809 12:05:49.063761 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.063770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.069302 (+ 5532us) server_negotiation.cc:407] Connection header received
0809 12:05:49.069572 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.069580 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.069835 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.070137 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:05:49.072419 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.073760 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.076882 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.078811 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.078949 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.079165 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.080807 (+ 1642us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.081007 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986,"thread_start_us":796,"threads_started":1}
I20260809 12:05:49.084870 17176 tablet_service.cc:1467] Processing CreateTablet for tablet 50b071123d144578b4afdeb890ac5ffe (DEFAULT_TABLE table=test-table [id=1b34965e7a3f4a59a517be689207edf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:05:49.084729 16975 tablet_service.cc:1467] Processing CreateTablet for tablet 50b071123d144578b4afdeb890ac5ffe (DEFAULT_TABLE table=test-table [id=1b34965e7a3f4a59a517be689207edf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:05:49.086457 17176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50b071123d144578b4afdeb890ac5ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:05:49.086463 16975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50b071123d144578b4afdeb890ac5ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:05:49.088699 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.059945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.0.147.67:43121)
0809 12:05:49.066963 (+ 7018us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.066972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.067039 (+ 67us) server_negotiation.cc:407] Connection header received
0809 12:05:49.067206 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.067218 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.067574 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.067933 (+ 359us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:05:49.069129 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.071096 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.075338 (+ 4242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.077609 (+ 2271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.077741 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.087920 (+ 10179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.088002 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.088208 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6764,"thread_start_us":6454,"threads_started":1}
I20260809 12:05:49.093739 17370 tablet_service.cc:1467] Processing CreateTablet for tablet 50b071123d144578b4afdeb890ac5ffe (DEFAULT_TABLE table=test-table [id=1b34965e7a3f4a59a517be689207edf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:05:49.097244 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50b071123d144578b4afdeb890ac5ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:05:49.103132 17499 tablet_bootstrap.cc:492] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: Bootstrap starting.
I20260809 12:05:49.107149 17500 tablet_bootstrap.cc:492] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98: Bootstrap starting.
I20260809 12:05:49.109611 17499 tablet_bootstrap.cc:654] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: Neither blocks nor log segments found. Creating new log.
I20260809 12:05:49.112108 17499 log.cc:824] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:05:49.113490 17500 tablet_bootstrap.cc:654] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98: Neither blocks nor log segments found. Creating new log.
I20260809 12:05:49.115340 17500 log.cc:824] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98: Log is configured to *not* fsync() on all Append() calls
I20260809 12:05:49.116765 17499 tablet_bootstrap.cc:492] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: No bootstrap required, opened a new log
I20260809 12:05:49.117036 17499 ts_tablet_manager.cc:1397] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.007s
I20260809 12:05:49.119807 17500 tablet_bootstrap.cc:492] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98: No bootstrap required, opened a new log
I20260809 12:05:49.120505 17500 ts_tablet_manager.cc:1397] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.009s
I20260809 12:05:49.120754 17503 tablet_bootstrap.cc:492] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: Bootstrap starting.
I20260809 12:05:49.125182 17503 tablet_bootstrap.cc:654] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: Neither blocks nor log segments found. Creating new log.
I20260809 12:05:49.126849 17503 log.cc:824] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: Log is configured to *not* fsync() on all Append() calls
I20260809 12:05:49.131124 17276 heartbeater.cc:503] Master 127.0.147.126:46201 was elected leader, sending a full tablet report...
I20260809 12:05:49.136519 17500 raft_consensus.cc:348] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } }
I20260809 12:05:49.136976 17503 tablet_bootstrap.cc:492] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: No bootstrap required, opened a new log
I20260809 12:05:49.136445 17499 raft_consensus.cc:348] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } }
I20260809 12:05:49.137058 17500 raft_consensus.cc:374] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:05:49.137217 17499 raft_consensus.cc:374] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:05:49.137259 17500 raft_consensus.cc:729] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa6404eb0b645f29565570c2ec12a98, State: Initialized, Role: FOLLOWER
I20260809 12:05:49.137462 17499 raft_consensus.cc:729] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8819bbedc77b492ba739c36f788fc0a0, State: Initialized, Role: FOLLOWER
I20260809 12:05:49.137771 17503 ts_tablet_manager.cc:1397] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.009s
I20260809 12:05:49.138513 17500 consensus_queue.cc:260] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } }
I20260809 12:05:49.138772 17499 consensus_queue.cc:260] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } }
I20260809 12:05:49.141623 17500 ts_tablet_manager.cc:1428] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98: Time spent starting tablet: real 0.021s user 0.012s sys 0.007s
I20260809 12:05:49.142333 17499 ts_tablet_manager.cc:1428] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: Time spent starting tablet: real 0.025s user 0.016s sys 0.004s
I20260809 12:05:49.151937 17503 raft_consensus.cc:348] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } }
I20260809 12:05:49.152415 17503 raft_consensus.cc:374] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:05:49.152590 17503 raft_consensus.cc:729] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a18508bbc84d0fa89b7d902220d793, State: Initialized, Role: FOLLOWER
I20260809 12:05:49.153384 17503 consensus_queue.cc:260] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } }
I20260809 12:05:49.154860 17472 heartbeater.cc:503] Master 127.0.147.126:46201 was elected leader, sending a full tablet report...
I20260809 12:05:49.155790 17503 ts_tablet_manager.cc:1428] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
W20260809 12:05:49.221398 17473 tablet.cc:2369] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:05:49.273777 17506 raft_consensus.cc:482] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:05:49.274132 17506 raft_consensus.cc:504] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } }
I20260809 12:05:49.276237 17506 leader_election.cc:288] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65a18508bbc84d0fa89b7d902220d793 (127.0.147.67:43121), 8819bbedc77b492ba739c36f788fc0a0 (127.0.147.66:39867)
I20260809 12:05:49.286893 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.276505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:40429 (local address 127.0.147.67:43121)
0809 12:05:49.276757 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.276764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.277938 (+ 1174us) server_negotiation.cc:407] Connection header received
0809 12:05:49.278138 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.278144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.278214 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.278313 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:05:49.279642 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.280977 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.284197 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.285662 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.285811 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.285866 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.286465 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.286564 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:05:49.287129 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.276693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43121 (local address 127.0.147.65:40429)
0809 12:05:49.277781 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.277807 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:05:49.277975 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:05:49.278339 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:05:49.278343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:05:49.278371 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:05:49.279417 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.279434 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.281182 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.281187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.284041 (+ 2854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.284054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.286054 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.286070 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.286284 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.286320 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:05:49.286423 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 12:05:49.286560 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"mutex_wait_us":1,"thread_start_us":719,"threads_started":1}
I20260809 12:05:49.287945 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.276624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:51037 (local address 127.0.147.66:39867)
0809 12:05:49.276918 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.276922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.279283 (+ 2361us) server_negotiation.cc:407] Connection header received
0809 12:05:49.279444 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.279449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.279573 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.279679 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:05:49.280769 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.282267 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.285172 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.286666 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.286821 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.286969 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.287482 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.287644 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 12:05:49.288908 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.277518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:39867 (local address 127.0.147.65:51037)
0809 12:05:49.279002 (+ 1484us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.279024 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:05:49.279306 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:05:49.279698 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:05:49.279703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:05:49.279734 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:05:49.280572 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.280588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.282416 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.282420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.284988 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:05:49.285003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.287055 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.287063 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:05:49.287270 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.287309 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:05:49.288215 (+ 906us) client_negotiation.cc:240] Negotiation successful
0809 12:05:49.288387 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1326,"mutex_wait_us":1,"thread_start_us":1068,"threads_started":1}
I20260809 12:05:49.288656 17398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50b071123d144578b4afdeb890ac5ffe" candidate_uuid: "caa6404eb0b645f29565570c2ec12a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a18508bbc84d0fa89b7d902220d793" is_pre_election: true
I20260809 12:05:49.289309 17398 raft_consensus.cc:2454] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caa6404eb0b645f29565570c2ec12a98 in term 0.
I20260809 12:05:49.290165 16910 leader_election.cc:302] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65a18508bbc84d0fa89b7d902220d793, caa6404eb0b645f29565570c2ec12a98; no voters:
I20260809 12:05:49.290372 17205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50b071123d144578b4afdeb890ac5ffe" candidate_uuid: "caa6404eb0b645f29565570c2ec12a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8819bbedc77b492ba739c36f788fc0a0" is_pre_election: true
I20260809 12:05:49.290822 17506 raft_consensus.cc:2781] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:05:49.291039 17506 raft_consensus.cc:482] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:05:49.291232 17506 raft_consensus.cc:3037] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:05:49.291119 17205 raft_consensus.cc:2454] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caa6404eb0b645f29565570c2ec12a98 in term 0.
I20260809 12:05:49.294207 17506 raft_consensus.cc:504] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } }
I20260809 12:05:49.294981 17506 leader_election.cc:288] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [CANDIDATE]: Term 1 election: Requested vote from peers 65a18508bbc84d0fa89b7d902220d793 (127.0.147.67:43121), 8819bbedc77b492ba739c36f788fc0a0 (127.0.147.66:39867)
I20260809 12:05:49.295367 17398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50b071123d144578b4afdeb890ac5ffe" candidate_uuid: "caa6404eb0b645f29565570c2ec12a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a18508bbc84d0fa89b7d902220d793"
I20260809 12:05:49.295496 17205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50b071123d144578b4afdeb890ac5ffe" candidate_uuid: "caa6404eb0b645f29565570c2ec12a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8819bbedc77b492ba739c36f788fc0a0"
I20260809 12:05:49.295634 17398 raft_consensus.cc:3037] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:05:49.295780 17205 raft_consensus.cc:3037] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:05:49.298498 17398 raft_consensus.cc:2454] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caa6404eb0b645f29565570c2ec12a98 in term 1.
I20260809 12:05:49.298532 17205 raft_consensus.cc:2454] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caa6404eb0b645f29565570c2ec12a98 in term 1.
I20260809 12:05:49.299125 16912 leader_election.cc:302] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8819bbedc77b492ba739c36f788fc0a0, caa6404eb0b645f29565570c2ec12a98; no voters:
I20260809 12:05:49.299568 17506 raft_consensus.cc:2781] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:05:49.300372 17506 raft_consensus.cc:686] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 1 LEADER]: Becoming Leader. State: Replica: caa6404eb0b645f29565570c2ec12a98, State: Running, Role: LEADER
I20260809 12:05:49.301012 17506 consensus_queue.cc:237] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } }
I20260809 12:05:49.306653 16753 catalog_manager.cc:5495] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 reported cstate change: term changed from 0 to 1, leader changed from <none> to caa6404eb0b645f29565570c2ec12a98 (127.0.147.65). New cstate: current_term: 1 leader_uuid: "caa6404eb0b645f29565570c2ec12a98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caa6404eb0b645f29565570c2ec12a98" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 } health_report { overall_health: UNKNOWN } } }
W20260809 12:05:49.335430 17279 tablet.cc:2369] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:05:49.361608 17072 tablet.cc:2369] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:05:49.377684 17494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.371179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57254 (local address 127.0.147.65:46125)
0809 12:05:49.371457 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.371462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.371477 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:05:49.371619 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.371623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.371740 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.371908 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:05:49.372379 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.373458 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.375191 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.375728 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.375867 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.376523 (+ 656us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:05:49.377030 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:05:49.377194 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.377301 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.377383 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:05:49.388481 17398 raft_consensus.cc:1261] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [term 1 FOLLOWER]: Refusing update from remote peer caa6404eb0b645f29565570c2ec12a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:05:49.388532 17205 raft_consensus.cc:1261] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [term 1 FOLLOWER]: Refusing update from remote peer caa6404eb0b645f29565570c2ec12a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:05:49.389528 17515 consensus_queue.cc:1035] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65a18508bbc84d0fa89b7d902220d793" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:05:49.390074 17506 consensus_queue.cc:1035] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8819bbedc77b492ba739c36f788fc0a0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 39867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:05:49.405490 17519 mvcc.cc:205] Tried to move back new op lower bound from 7316591203882520576 to 7316591203540885504. Current Snapshot: MvccSnapshot[applied={T|T < 7316591203882520576}]
I20260809 12:05:49.408922 17521 mvcc.cc:205] Tried to move back new op lower bound from 7316591203882520576 to 7316591203540885504. Current Snapshot: MvccSnapshot[applied={T|T < 7316591203882520576}]
I20260809 12:05:49.426071 16752 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:05:49.426612 16752 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:05:49.431460 16752 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03ecd58cd2e24d1cb950ddd1e2ba15f4: Sending DeleteTablet for 3 replicas of tablet 50b071123d144578b4afdeb890ac5ffe
I20260809 12:05:49.432952 17370 tablet_service.cc:1514] Processing DeleteTablet for tablet 50b071123d144578b4afdeb890ac5ffe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:05:49 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:47220
I20260809 12:05:49.432976 17176 tablet_service.cc:1514] Processing DeleteTablet for tablet 50b071123d144578b4afdeb890ac5ffe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:05:49 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:55034
I20260809 12:05:49.433054 16975 tablet_service.cc:1514] Processing DeleteTablet for tablet 50b071123d144578b4afdeb890ac5ffe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:05:49 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:57248
I20260809 12:05:49.436180 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 16871
I20260809 12:05:49.445816 17534 tablet_replica.cc:330] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: stopping tablet replica
I20260809 12:05:49.447899 17534 raft_consensus.cc:2229] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:05:49.449826 17534 raft_consensus.cc:2258] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:05:49.452317 17535 tablet_replica.cc:330] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: stopping tablet replica
I20260809 12:05:49.454763 17534 ts_tablet_manager.cc:1905] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:05:49.455327 17535 raft_consensus.cc:2229] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:05:49.456046 17535 raft_consensus.cc:2258] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:05:49.461740 17535 ts_tablet_manager.cc:1905] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 12:05:49 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:05:49.464039 17536 tablet_replica.cc:330] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98: stopping tablet replica
I20260809 12:05:49.466024 17536 raft_consensus.cc:2229] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:05:49.468773 17536 raft_consensus.cc:2258] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:05:49.471163 17535 ts_tablet_manager.cc:1918] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:05:49.471521 17535 log.cc:1196] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/50b071123d144578b4afdeb890ac5ffe
I20260809 12:05:49.472519 17536 ts_tablet_manager.cc:1905] T 50b071123d144578b4afdeb890ac5ffe P caa6404eb0b645f29565570c2ec12a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:05:49.472810 17535 ts_tablet_manager.cc:1939] T 50b071123d144578b4afdeb890ac5ffe P 8819bbedc77b492ba739c36f788fc0a0: Deleting consensus metadata
I20260809 12:05:49.475222 16730 catalog_manager.cc:4841] TS 8819bbedc77b492ba739c36f788fc0a0 (127.0.147.66:39867): tablet 50b071123d144578b4afdeb890ac5ffe (table test-table [id=1b34965e7a3f4a59a517be689207edf8]) successfully deleted
I20260809 12:05:49.476552 17534 ts_tablet_manager.cc:1918] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:05:49.476791 17534 log.cc:1196] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/50b071123d144578b4afdeb890ac5ffe
I20260809 12:05:49.477401 17534 ts_tablet_manager.cc:1939] T 50b071123d144578b4afdeb890ac5ffe P 65a18508bbc84d0fa89b7d902220d793: Deleting consensus metadata
I20260809 12:05:49.477784 17494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.437430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.0.147.65:46125)
0809 12:05:49.437650 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.437659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.450889 (+ 13230us) server_negotiation.cc:407] Connection header received
0809 12:05:49.451509 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.451515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.451640 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.451731 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:05:49.453796 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.455059 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.456986 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.457722 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.457858 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.467172 (+ 9314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:05:49.467189 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:05:49.467208 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:05:49.467253 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:05:49.473171 (+ 5918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:49.474302 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:49.474320 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:49.474324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:49.474633 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:49.475257 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:49.475266 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:49.475272 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:49.475867 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:05:49.476894 (+ 1027us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.477128 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.477365 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 12:05:49.479377 17066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:05:49.479902 16727 catalog_manager.cc:4841] TS 65a18508bbc84d0fa89b7d902220d793 (127.0.147.67:43121): tablet 50b071123d144578b4afdeb890ac5ffe (table test-table [id=1b34965e7a3f4a59a517be689207edf8]) successfully deleted
I20260809 12:05:49.582070 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 16871
I20260809 12:05:49.606902 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 17081
W20260809 12:05:49.606805 16730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.147.65:46125 (error 108)
I20260809 12:05:49.609448 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.608310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46125 (local address 127.0.0.1:57274)
0809 12:05:49.609046 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.609238 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:46125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":467,"threads_started":1}
W20260809 12:05:49.609699 16730 catalog_manager.cc:4584] TS caa6404eb0b645f29565570c2ec12a98 (127.0.147.65:46125): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50b071123d144578b4afdeb890ac5ffe: Network error: Client connection negotiation failed: client connection to 127.0.147.65:46125: connect: Connection refused (error 111)
Aug 09 12:05:49 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:05:49.626236 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.608014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.0.147.66:39867)
0809 12:05:49.608193 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.608198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.608216 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:05:49.608380 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.608384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.608445 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.608590 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:05:49.610315 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.611025 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.612598 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.613007 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.613147 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.620528 (+ 7381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:05:49.620543 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:05:49.620564 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:05:49.620604 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:05:49.623339 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:49.624126 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:49.624133 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:49.624136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:49.624264 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:49.624652 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:49.624657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:49.624659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:49.625007 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:05:49.625325 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.625638 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.625888 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:05:49.627085 17268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:05:49.663383 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.662821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46125 (local address 127.0.0.1:57288)
0809 12:05:49.663053 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.663188 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:46125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 12:05:49.664582 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.664129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46125 (local address 127.0.0.1:57296)
0809 12:05:49.664260 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.664330 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:46125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 12:05:49.718142 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 17081
I20260809 12:05:49.725152 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.724468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46125 (local address 127.0.0.1:57310)
0809 12:05:49.724740 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.724916 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:46125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 12:05:49.726454 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.725986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46125 (local address 127.0.0.1:57324)
0809 12:05:49.726175 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.726270 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:46125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 12:05:49.744117 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 17287
Aug 09 12:05:49 dist-test-slave-sb2z krb5kdc[16054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:05:49.762396 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.745004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.0.147.67:43121)
0809 12:05:49.745239 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 12:05:49.745245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:05:49.745264 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:05:49.745320 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:05:49.745324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:05:49.745433 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:05:49.745582 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:05:49.747384 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.748304 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.749673 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:05:49.750125 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:05:49.750271 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:05:49.755089 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:05:49.755106 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:05:49.755124 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:05:49.755175 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:05:49.758989 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:49.759850 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:49.759867 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:49.759869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:49.760117 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:05:49.760523 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:05:49.760530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:05:49.760533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:05:49.760996 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:05:49.761435 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0809 12:05:49.761715 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 12:05:49.761953 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:05:49.763072 17467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:05:49.807703 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.807109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46125 (local address 127.0.0.1:57334)
0809 12:05:49.807361 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.807469 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:46125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 12:05:49.809028 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.808592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46125 (local address 127.0.0.1:57348)
0809 12:05:49.808802 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.808887 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:46125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 12:05:49.852711 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 17287
I20260809 12:05:49.876814 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 16707
I20260809 12:05:49.878010 16802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:05:49.992496 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:05:49.991452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46125 (local address 127.0.0.1:57360)
0809 12:05:49.992097 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0809 12:05:49.992301 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:46125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293}
I20260809 12:05:49.992681 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 16707
I20260809 12:05:50.027516 589 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 12:05:50.372706 589 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 12:05:50.373097 589 mini_ranger.cc:66] Stopping Ranger...
I20260809 12:05:50.729249 589 mini_ranger.cc:68] Stopped Ranger
2026-08-09 12:05:50.729 UTC [16105] LOG: received smart shutdown request
2026-08-09 12:05:50.733 UTC [16105] LOG: background worker "logical replication launcher" (PID 16118) exited with exit code 1
2026-08-09 12:05:50.733 UTC [16113] LOG: shutting down
2026-08-09 12:05:50.880 UTC [16105] LOG: database system is shut down
2026-08-09T12:05:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52349 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:05:51 dist-test-slave-sb2z krb5kdc[17555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:05:51 dist-test-slave-sb2z krb5kdc[17555](info): set up 2 sockets
Aug 09 12:05:51 dist-test-slave-sb2z krb5kdc[17555](info): commencing operation
krb5kdc: starting...
W20260809 12:05:52.364670 589 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:05:52 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:05:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:05:52Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.147.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.147.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/rangeradmin_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/rangeradmin_127.0.147.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.147.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.147.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/rangerlookup_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/rangerlookup_127.0.147.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.147.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.147.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/HTTP_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/HTTP_127.0.147.84@KRBTEST.COM.keytab.
I20260809 12:05:52.773754 589 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task3WUkck/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/postgres -l logfile start
2026-08-09 12:05:55.112 UTC [17598] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 12:05:55.112 UTC [17598] LOG: listening on IPv4 address "127.0.147.84", port 34617
2026-08-09 12:05:55.115 UTC [17598] LOG: listening on Unix socket "/tmp/.s.PGSQL.34617"
2026-08-09 12:05:55.139 UTC [17600] LOG: database system was shut down at 2026-08-09 12:05:53 UTC
2026-08-09 12:05:55.143 UTC [17598] LOG: database system is ready to accept connections
I20260809 12:05:56.305258 589 mini_postgres.cc:96] Postgres bound to 34617
2026-08-09 12:05:56.330 UTC [17611] FATAL: database "slave" does not exist
127.0.147.84:34617 - accepting connections
I20260809 12:05:56.330874 589 mini_ranger.cc:161] Starting Ranger...
I20260809 12:05:56.361650 589 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 12:05:56.807063 589 mini_ranger.cc:87] Created ranger Postgres database
I20260809 12:05:56.807199 589 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-admin
2026-08-09 12:05:57,119 [I] DB FLAVOR :POSTGRES
2026-08-09 12:05:57,120 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:05:57,120 [I] Checking connection..
2026-08-09 12:05:57,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 12:05:57,378 [I] Checking connection passed.
2026-08-09 12:05:57,378 [I] --------- Verifying version history table ---------
2026-08-09 12:05:57,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:05:57,647 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:05:57,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:05:57,906 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:05:57,906 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 12:05:57,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 12:05:58.181 UTC [17699] WARNING: there is no transaction in progress
2026-08-09 12:05:58,189 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 12:05:58,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:05:58,442 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 12:05:58,442 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:05:58,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 12:05:58,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 12:05:58,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-09 12:05:59,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:05:59,466 [I] Table x_portal_user does not exist in database ranger
2026-08-09 12:05:59,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:05:59,725 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 12:05:59,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:05:59,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:06:00,237 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 12:06:00,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 12:06:01.137 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.153 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.171 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.185 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.203 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.219 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.274 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.281 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.295 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.306 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.316 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.330 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.339 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.345 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.352 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01.358 UTC [17873] WARNING: there is no transaction in progress
2026-08-09 12:06:01,950 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 12:06:01,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 12:06:02,184 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 12:06:02,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:06:02,448 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 12:06:02,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:06:02,710 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 12:06:02,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:06:02,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:06:03,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-09 12:06:03,467 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 12:06:03,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 12:06:03,734 [I] DB_PATCHES have already been applied
I20260809 12:06:03.740198 589 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-admin:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task3WUkck/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 12:06:03.740285 589 mini_ranger.cc:193] Using host: 127.0.147.84
I20260809 12:06:03.763118 589 mini_ranger.cc:227] Ranger admin URL: http://127.0.147.84:58561
Aug 09, 2026 12:06:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:06:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:06:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:06:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:06:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:06:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:06:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 12:06:05.120782 589 mini_ranger.cc:160] Time spent starting Ranger: real 8.790s user 0.007s sys 0.058s
Aug 09, 2026 12:06:25 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:06:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 12:06:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:06:25 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:06:26 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 12:06:27.442772 589 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.147.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.147.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/rangerkms_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.147.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/rangerkms_127.0.147.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.147.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.147.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.147.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/HTTP_127.0.147.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.147.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/HTTP_127.0.147.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 12:06:27 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277187, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 12:06:27.661249 589 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 12:06:27.688546 589 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 12:06:28.494616 589 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 12:06:28.494716 589 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-kms
I20260809 12:06:28.494748 589 mini_ranger_kms.cc:227] Using postgres at 127.0.147.84:34617
2026-08-09 12:06:28,899 [I] DB FLAVOR :POSTGRES
2026-08-09 12:06:28,899 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:06:28,899 [I] Checking connection
2026-08-09 12:06:28,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 12:06:29,166 [I] connection success
2026-08-09 12:06:29,167 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 12:06:29,167 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 12:06:29,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 12:06:29,420 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 12:06:29,420 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:06:29,420 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 12:06:29,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.147.84:34617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 12:06:29,674 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 12:06:30.183979 589 mini_ranger_kms.cc:310] Created kms service
I20260809 12:06:30.350662 589 mini_ranger_kms.cc:326] Created kudu user
I20260809 12:06:30.396710 589 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 12:06:30 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277187, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:30.837150 589 mini_ranger_kms.cc:384] Added ranger policy
I20260809 12:06:30.837496 589 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-kms:/tmp/dist-test-task3WUkck/build/asan/bin/postgresql.jar:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task3WUkck/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 12:06:30.837548 589 mini_ranger_kms.cc:240] Using host: 127.0.147.84
I20260809 12:06:30.866501 589 mini_ranger_kms.cc:276] Ranger KMS PID: 18138
I20260809 12:06:30.866639 589 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.147.84:48319
Aug 09, 2026 12:06:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:06:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:06:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:06:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:06:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:06:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 12:06:33 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277193, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.147.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 12:06:33 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277193, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.147.84@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:33.649735 589 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.988s user 0.079s sys 0.365s
I20260809 12:06:33.650000 589 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 12:06:33.650183 589 mini_ranger_kms.cc:401] 127.0.147.84:48319/kms/v1/keys
Aug 09 12:06:33 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277193, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.147.84@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
Aug 09 12:06:36 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:06:36.260504 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45935
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:45935
--ranger_config_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:48319/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:06:36.594985 18215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:36.595367 18215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:36.595568 18215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:36.602701 18215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:06:36.602844 18215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:36.602946 18215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:36.603042 18215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:06:36.603132 18215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:06:36.613083 18215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.147.84:48319/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:45935
--ranger_config_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:36.614349 18215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:36.615677 18215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:36.627589 18227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:36.627617 18230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:36.627664 18226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:36.628576 18215 server_base.cc:1034] running on GCE node
I20260809 12:06:36.629315 18215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:36.630640 18215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:36.631860 18215 hybrid_clock.cc:648] HybridClock initialized: now 1786277196631830 us; error 44 us; skew 500 ppm
I20260809 12:06:36.632213 18215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:36 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:36.639216 18215 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:06:36.640652 18215 webserver.cc:460] Webserver started at http://127.0.147.126:37155/ using document root <none> and password file <none>
I20260809 12:06:36.641297 18215 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:36.641439 18215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:36.641744 18215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:06:36 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:36.712435 18215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "a439e054f8c547479e8c9a4baa1dc468"
format_stamp: "Formatted at 2026-08-09 12:06:36 on dist-test-slave-sb2z"
server_key: "beadb45b0c5ca7b515bbcec295a9ccbc"
server_key_iv: "576eee4f79ee819a3870430f41358871"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:36.713316 18215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "a439e054f8c547479e8c9a4baa1dc468"
format_stamp: "Formatted at 2026-08-09 12:06:36 on dist-test-slave-sb2z"
server_key: "beadb45b0c5ca7b515bbcec295a9ccbc"
server_key_iv: "576eee4f79ee819a3870430f41358871"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:36.718837 18215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:06:36.738252 18240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:36.739837 18215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:06:36.740056 18215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "a439e054f8c547479e8c9a4baa1dc468"
format_stamp: "Formatted at 2026-08-09 12:06:36 on dist-test-slave-sb2z"
server_key: "beadb45b0c5ca7b515bbcec295a9ccbc"
server_key_iv: "576eee4f79ee819a3870430f41358871"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:36.740270 18215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:36.762284 18215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:36.766417 18215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:36.766762 18215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:36.787634 18215 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:45935
I20260809 12:06:36.787654 18314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:45935 every 8 connection(s)
I20260809 12:06:36.789050 18215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:06:36.790863 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 18215
I20260809 12:06:36.791103 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:06:36.792061 589 external_mini_cluster.cc:1428] Setting key 94879e7126768d9f3f91e4e8bf83e696
I20260809 12:06:36.794446 18315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:36.810909 18315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Bootstrap starting.
I20260809 12:06:36.814563 18315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:36.815642 18315 log.cc:824] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:36.818786 18315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: No bootstrap required, opened a new log
Aug 09 12:06:36 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:36.828728 18315 raft_consensus.cc:348] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } }
I20260809 12:06:36.829172 18315 raft_consensus.cc:374] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:36.829284 18315 raft_consensus.cc:729] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a439e054f8c547479e8c9a4baa1dc468, State: Initialized, Role: FOLLOWER
I20260809 12:06:36.829957 18315 consensus_queue.cc:260] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } }
I20260809 12:06:36.830278 18315 raft_consensus.cc:388] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:06:36.830430 18315 raft_consensus.cc:482] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:06:36.830647 18315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:36.832252 18319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:36.794401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42652 (local address 127.0.147.126:45935)
0809 12:06:36.805801 (+ 11400us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:36.805812 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:36.808161 (+ 2349us) server_negotiation.cc:407] Connection header received
0809 12:06:36.810764 (+ 2603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:36.810796 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:36.811296 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:36.811777 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:36.813566 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:36.815606 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:36.817203 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:36.818250 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:36.818441 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:36.823496 (+ 5055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:36.823514 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:36.823557 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:36.823612 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:36.827886 (+ 4274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:36.828727 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:36.828734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:36.828739 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:36.828881 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:36.829394 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:36.829399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:36.829403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:36.829919 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:36.830167 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:36.830712 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 12:06:36.831154 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11163,"thread_start_us":10942,"threads_started":1}
I20260809 12:06:36.833158 18315 raft_consensus.cc:504] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } }
I20260809 12:06:36.833745 18315 leader_election.cc:302] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a439e054f8c547479e8c9a4baa1dc468; no voters:
I20260809 12:06:36.834414 18315 leader_election.cc:288] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:06:36.834532 18322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:06:36.835235 18322 raft_consensus.cc:686] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 1 LEADER]: Becoming Leader. State: Replica: a439e054f8c547479e8c9a4baa1dc468, State: Running, Role: LEADER
I20260809 12:06:36.835898 18322 consensus_queue.cc:237] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } }
I20260809 12:06:36.837808 18315 sys_catalog.cc:564] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:06:36.838558 18323 sys_catalog.cc:455] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a439e054f8c547479e8c9a4baa1dc468. Latest consensus state: current_term: 1 leader_uuid: "a439e054f8c547479e8c9a4baa1dc468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } } }
I20260809 12:06:36.839243 18323 sys_catalog.cc:458] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:36.838762 18324 sys_catalog.cc:455] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a439e054f8c547479e8c9a4baa1dc468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } } }
I20260809 12:06:36.840068 18324 sys_catalog.cc:458] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:36.841063 18328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:06:36.845575 18328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:06:36.855321 18328 catalog_manager.cc:1347] Generated new cluster ID: d6845fd86f7d414a80265d292a2abfa5
I20260809 12:06:36.855443 18328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:06:36.861320 18315 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 12:06:36.886440 18328 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:06:36.888095 18328 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:06:36.913834 18328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Generated new TSK 0
I20260809 12:06:36.914732 18328 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:06:37.673902 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:48319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45935
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:06:37 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277197, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 12:06:38.114992 18374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:38.115360 18374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:38.115597 18374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:38.127144 18374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:38.127341 18374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:38.127586 18374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:06:38.143227 18374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.147.84:48319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:38.144881 18374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:38.146588 18374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:38.167586 18389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:38.167979 18391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:38.167696 18386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:38.172190 18374 server_base.cc:1034] running on GCE node
I20260809 12:06:38.172914 18374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:38.173936 18374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:38.175174 18374 hybrid_clock.cc:648] HybridClock initialized: now 1786277198175114 us; error 55 us; skew 500 ppm
I20260809 12:06:38.175514 18374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:38 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:38.179277 18374 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:06:38.181358 18374 webserver.cc:460] Webserver started at http://127.0.147.65:41915/ using document root <none> and password file <none>
I20260809 12:06:38.182425 18374 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:38.182590 18374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:38.183035 18374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:06:38 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:38.213083 18374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "24f9e917b28346afa97640ecb324e517"
format_stamp: "Formatted at 2026-08-09 12:06:38 on dist-test-slave-sb2z"
server_key: "ff6bbd791d34da47ab690840076c5d33"
server_key_iv: "24884b714f9aa6127515da4d92a7a218"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:38.214795 18374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "24f9e917b28346afa97640ecb324e517"
format_stamp: "Formatted at 2026-08-09 12:06:38 on dist-test-slave-sb2z"
server_key: "ff6bbd791d34da47ab690840076c5d33"
server_key_iv: "24884b714f9aa6127515da4d92a7a218"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:38.226189 18374 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.006s sys 0.003s
I20260809 12:06:38.260185 18406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:38.262130 18374 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.007s
I20260809 12:06:38.262411 18374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "24f9e917b28346afa97640ecb324e517"
format_stamp: "Formatted at 2026-08-09 12:06:38 on dist-test-slave-sb2z"
server_key: "ff6bbd791d34da47ab690840076c5d33"
server_key_iv: "24884b714f9aa6127515da4d92a7a218"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:38.262773 18374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:38.283262 18374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:38.289124 18374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:38.289529 18374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:38.292462 18374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:38.300359 18374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:38.300504 18374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:38.300655 18374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:38.300740 18374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:06:38 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277197, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:38.429308 18374 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:42545
I20260809 12:06:38.429350 18563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:42545 every 8 connection(s)
I20260809 12:06:38.431015 18374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:06:38.436023 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 18374
I20260809 12:06:38.436254 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:06:38.437143 589 external_mini_cluster.cc:1428] Setting key d5419753371ef06d8143226a2d467719
Aug 09 12:06:38 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:38.460279 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:38.435317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:50381 (local address 127.0.147.126:45935)
0809 12:06:38.438412 (+ 3095us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:38.438421 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:38.438452 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:06:38.442840 (+ 4388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:38.442846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:38.442938 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:38.443031 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:38.445069 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:38.446146 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:38.448404 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:38.448981 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:38.449067 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:38.453403 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:38.453409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:38.453414 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:38.453471 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:38.456828 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:38.457594 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:38.457599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:38.457603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:38.457744 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:38.458207 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:38.458212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:38.458215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:38.458518 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:38.458795 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:38.459572 (+ 777us) server_negotiation.cc:299] Negotiation successful
0809 12:06:38.459898 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2965,"thread_start_us":3593,"threads_started":1}
I20260809 12:06:38.461279 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:38.434647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45935 (local address 127.0.147.65:50381)
0809 12:06:38.436961 (+ 2314us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:38.437019 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:38.442427 (+ 5408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:38.443455 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:38.443466 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:38.444079 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:38.444873 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:38.444897 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:38.446334 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:38.446339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:38.448245 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:38.448260 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:38.449383 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:38.449389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:38.449658 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:38.450407 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:38.450446 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:38.453158 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:38.456976 (+ 3818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:38.456985 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:38.457001 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:38.457447 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:38.457867 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:38.457874 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:38.457877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:38.458084 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:38.458765 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:38.458773 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:38.459165 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 12:06:38.459433 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 12:06:38.460001 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2012,"thread_start_us":3367,"threads_started":1}
I20260809 12:06:38.462927 18564 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45935
I20260809 12:06:38.463260 18564 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:38.464054 18564 heartbeater.cc:511] Master 127.0.147.126:45935 requested a full tablet report, sending...
I20260809 12:06:38.466122 18265 ts_manager.cc:194] Registered new tserver with Master: 24f9e917b28346afa97640ecb324e517 (127.0.147.65:42545)
I20260809 12:06:38.467703 18265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:50381
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:06:38.514452 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:48319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45935
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:06:38 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277197, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
W20260809 12:06:38.967736 18576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:38.968171 18576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:38.968345 18576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:38.981359 18576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:38.981525 18576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:38.981787 18576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:06:38.999400 18576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.147.84:48319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:39.001595 18576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:39.003661 18576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:39.022549 18588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:39.023479 18590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:39.026803 18600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:39.028146 18576 server_base.cc:1034] running on GCE node
I20260809 12:06:39.028869 18576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:39.029940 18576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:39.031172 18576 hybrid_clock.cc:648] HybridClock initialized: now 1786277199031134 us; error 37 us; skew 500 ppm
I20260809 12:06:39.031584 18576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:39 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:39.034922 18576 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:06:39.036691 18576 webserver.cc:460] Webserver started at http://127.0.147.66:36141/ using document root <none> and password file <none>
I20260809 12:06:39.037463 18576 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:39.037601 18576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:39.037958 18576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:06:39 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:39.056445 18576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "40b39750cb1e4fee9468127cc1c824c0"
format_stamp: "Formatted at 2026-08-09 12:06:39 on dist-test-slave-sb2z"
server_key: "eb82dc4ef176b3c8a1d92d253591413b"
server_key_iv: "bf97fda5c3610b160c6632b9e544ed78"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:39.057711 18576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "40b39750cb1e4fee9468127cc1c824c0"
format_stamp: "Formatted at 2026-08-09 12:06:39 on dist-test-slave-sb2z"
server_key: "eb82dc4ef176b3c8a1d92d253591413b"
server_key_iv: "bf97fda5c3610b160c6632b9e544ed78"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:39.065178 18576 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260809 12:06:39.082520 18613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:39.084427 18576 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:06:39.084621 18576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "40b39750cb1e4fee9468127cc1c824c0"
format_stamp: "Formatted at 2026-08-09 12:06:39 on dist-test-slave-sb2z"
server_key: "eb82dc4ef176b3c8a1d92d253591413b"
server_key_iv: "bf97fda5c3610b160c6632b9e544ed78"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:39.084908 18576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:39.103202 18576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:39.107229 18576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:39.107604 18576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:39.108623 18576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:39.110162 18576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:39.110239 18576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:39.110381 18576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:39.110435 18576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:39.148247 18576 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:36395
I20260809 12:06:39.148298 18763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:36395 every 8 connection(s)
I20260809 12:06:39.149613 18576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:06:39.158625 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 18576
I20260809 12:06:39.158818 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:06:39.159410 589 external_mini_cluster.cc:1428] Setting key c1a8f664db5c99e28bf3070f1fbb6b11
Aug 09 12:06:39 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:39.171591 18768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:39.152438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:53977 (local address 127.0.147.126:45935)
0809 12:06:39.152953 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:39.152958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:39.154053 (+ 1095us) server_negotiation.cc:407] Connection header received
0809 12:06:39.155997 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:39.156003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:39.156132 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:39.156257 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:39.157813 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.158953 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.161135 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.161736 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.161868 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:39.165316 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:39.165321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:39.165325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:39.165355 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:39.168220 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:39.169082 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:39.169086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:39.169089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:39.169211 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:39.169767 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:39.169771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:39.169773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:39.170054 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:39.170221 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:39.170939 (+ 718us) server_negotiation.cc:299] Negotiation successful
0809 12:06:39.171186 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":277,"threads_started":1}
I20260809 12:06:39.172727 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:39.153092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45935 (local address 127.0.147.66:53977)
0809 12:06:39.153889 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:39.153929 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:39.155725 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:39.156503 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:39.156518 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:39.156934 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:39.157604 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:39.157635 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.159149 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.159156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:39.160962 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:39.160974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.161952 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.161959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:39.162201 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:39.162891 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:39.162913 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:39.165098 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:39.168413 (+ 3315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:39.168428 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:39.168441 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:39.168939 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:39.169326 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:39.169332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:39.169335 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:39.169629 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:39.170197 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:39.170205 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:39.170542 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 12:06:39.170822 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 12:06:39.171348 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":368,"threads_started":1}
I20260809 12:06:39.174175 18764 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45935
I20260809 12:06:39.174499 18764 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:39.175237 18764 heartbeater.cc:511] Master 127.0.147.126:45935 requested a full tablet report, sending...
I20260809 12:06:39.177177 18265 ts_manager.cc:194] Registered new tserver with Master: 40b39750cb1e4fee9468127cc1c824c0 (127.0.147.66:36395)
I20260809 12:06:39.178378 18265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:53977
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:06:39.231231 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:48319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45935
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:06:39.470938 18564 heartbeater.cc:503] Master 127.0.147.126:45935 was elected leader, sending a full tablet report...
W20260809 12:06:39.580648 18772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:39.580967 18772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:39.581149 18772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:39.589799 18772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:39.589903 18772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:39.590098 18772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:06:39.599861 18772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.147.84:48319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:39.601132 18772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:39.602368 18772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:39.615514 18781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:39.615747 18780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:39.615581 18783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:39.616686 18772 server_base.cc:1034] running on GCE node
I20260809 12:06:39.617300 18772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:39.618255 18772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:39.619481 18772 hybrid_clock.cc:648] HybridClock initialized: now 1786277199619427 us; error 64 us; skew 500 ppm
I20260809 12:06:39.619851 18772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:39 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:39.622967 18772 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:06:39.624506 18772 webserver.cc:460] Webserver started at http://127.0.147.67:36395/ using document root <none> and password file <none>
I20260809 12:06:39.625216 18772 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:39.625365 18772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:39.625670 18772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:06:39 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:39.641278 18772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "abafa36ef7c148f8b7cef4de1bfee866"
format_stamp: "Formatted at 2026-08-09 12:06:39 on dist-test-slave-sb2z"
server_key: "3da3672505aefac93c902af78da94b3d"
server_key_iv: "828bbd1d08919e1b55894d502dffdc78"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:39.642119 18772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "abafa36ef7c148f8b7cef4de1bfee866"
format_stamp: "Formatted at 2026-08-09 12:06:39 on dist-test-slave-sb2z"
server_key: "3da3672505aefac93c902af78da94b3d"
server_key_iv: "828bbd1d08919e1b55894d502dffdc78"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:39.647949 18772 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:06:39.662415 18792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:39.664091 18772 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:06:39.664265 18772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "abafa36ef7c148f8b7cef4de1bfee866"
format_stamp: "Formatted at 2026-08-09 12:06:39 on dist-test-slave-sb2z"
server_key: "3da3672505aefac93c902af78da94b3d"
server_key_iv: "828bbd1d08919e1b55894d502dffdc78"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:39.664472 18772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:39.690277 18772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:39.693993 18772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:39.694247 18772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:39.695127 18772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:39.696525 18772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:39.696657 18772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:39.696796 18772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:39.696941 18772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:39.733353 18772 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:34461
I20260809 12:06:39.733373 18966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:34461 every 8 connection(s)
I20260809 12:06:39.734731 18772 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:06:39.736559 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 18772
I20260809 12:06:39.736791 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:06:39.737332 589 external_mini_cluster.cc:1428] Setting key 17894d0f2f84d0e316ba00dda7836117
Aug 09 12:06:39 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:39.756152 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:39.737665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:55573 (local address 127.0.147.126:45935)
0809 12:06:39.738592 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:39.738620 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:39.739417 (+ 797us) server_negotiation.cc:407] Connection header received
0809 12:06:39.741633 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:39.741639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:39.741721 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:39.741852 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:39.743369 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.744252 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.745914 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.746376 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.746513 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:39.749947 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:39.749952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:39.749956 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:39.750025 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:39.753282 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:39.754013 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:39.754017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:39.754019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:39.754158 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:39.754583 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:39.754586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:39.754588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:39.754856 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:39.754969 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:39.755720 (+ 751us) server_negotiation.cc:299] Negotiation successful
0809 12:06:39.755879 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":558,"threads_started":1}
I20260809 12:06:39.757128 18973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:39.738312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45935 (local address 127.0.147.67:55573)
0809 12:06:39.739245 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:39.739308 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:39.741366 (+ 2058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:39.742097 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:39.742105 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:39.742579 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:39.743196 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:39.743215 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.744420 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.744425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:39.745759 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:39.745779 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.746608 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.746618 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:39.746862 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:39.747495 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:39.747517 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:39.749735 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:39.753424 (+ 3689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:39.753431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:39.753444 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:39.753880 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:39.754260 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:39.754264 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:39.754266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:39.754476 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:39.754965 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:39.754970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:39.755266 (+ 296us) client_negotiation.cc:769] Sending connection context
0809 12:06:39.755633 (+ 367us) client_negotiation.cc:240] Negotiation successful
0809 12:06:39.756030 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":389,"threads_started":1}
I20260809 12:06:39.758541 18967 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45935
I20260809 12:06:39.758862 18967 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:39.759706 18967 heartbeater.cc:511] Master 127.0.147.126:45935 requested a full tablet report, sending...
I20260809 12:06:39.761155 18265 ts_manager.cc:194] Registered new tserver with Master: abafa36ef7c148f8b7cef4de1bfee866 (127.0.147.67:34461)
I20260809 12:06:39.761906 18265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:55573
I20260809 12:06:39.765591 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:06:39.767277 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 18374
Aug 09 12:06:39 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:06:39.784513 18976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:39.768401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.0.147.65:42545)
0809 12:06:39.769216 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:39.769222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:39.769245 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:06:39.769300 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:39.769306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:39.769515 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:39.769707 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:39.771245 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.772332 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.773927 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.774570 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.774704 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:39.778977 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:39.778987 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:39.779000 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:39.779033 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:39.781987 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:39.782596 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:39.782602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:39.782604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:39.782743 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:39.783074 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:39.783077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:39.783079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:39.783384 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:39.783794 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:39.783971 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 12:06:39.784210 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":426,"threads_started":1}
I20260809 12:06:39.785820 18558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:39.872886 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 18374
I20260809 12:06:39.901932 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 18576
Aug 09 12:06:39 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:06:39.923205 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:39.902813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.0.147.66:36395)
0809 12:06:39.903367 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:39.903373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:39.903400 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:06:39.903458 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:39.903464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:39.903751 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:39.903920 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:39.905292 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.906303 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.907760 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:39.908307 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:39.908456 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:39.917544 (+ 9088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:39.917555 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:39.917569 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:39.917608 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:39.920272 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:39.920992 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:39.921000 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:39.921002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:39.921139 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:39.921526 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:39.921535 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:39.921538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:39.921903 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:39.922224 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:39.922583 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 12:06:39.922858 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":296,"threads_started":1}
I20260809 12:06:39.924546 18757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:40.006561 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 18576
I20260809 12:06:40.028249 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 18772
Aug 09 12:06:40 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:06:40.045086 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:40.029391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38832 (local address 127.0.147.67:34461)
0809 12:06:40.030101 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:40.030108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:40.030139 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:06:40.030207 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:40.030214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:40.030418 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:40.030566 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:40.031850 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:40.032844 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:40.033999 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:40.034534 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:40.034627 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:40.039062 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:40.039072 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:40.039086 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:40.039120 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:40.042346 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:40.042949 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:40.042955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:40.042957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:40.043098 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:40.043458 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:40.043462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:40.043463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:40.043864 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:40.044168 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:40.044478 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 12:06:40.044670 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":402,"threads_started":1}
I20260809 12:06:40.046386 18961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:40.129799 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 18772
I20260809 12:06:40.154147 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 18215
I20260809 12:06:40.155594 18307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:40.262816 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 18215
I20260809 12:06:40.297299 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45935
--webserver_interface=127.0.147.126
--webserver_port=37155
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:45935
--ranger_config_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:48319/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.147.84:48319/kms with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:06:40.630343 18984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:40.630610 18984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:40.630800 18984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:40.638019 18984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:06:40.638134 18984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:40.638240 18984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:40.638339 18984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:06:40.638425 18984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:06:40.648478 18984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.147.84:48319/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:45935
--ranger_config_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=37155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:40.649735 18984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:40.651084 18984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:40.662973 18996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:40.663074 18994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:40.663118 18992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:40.663800 18984 server_base.cc:1034] running on GCE node
I20260809 12:06:40.664616 18984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:40.666031 18984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:40.667266 18984 hybrid_clock.cc:648] HybridClock initialized: now 1786277200667229 us; error 54 us; skew 500 ppm
I20260809 12:06:40.667691 18984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:40 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:40.670876 18984 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:06:40.672371 18984 webserver.cc:460] Webserver started at http://127.0.147.126:37155/ using document root <none> and password file <none>
I20260809 12:06:40.672971 18984 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:40.673081 18984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:06:40 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:40.748225 18984 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 12:06:40.751217 19008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:40.752717 18984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:06:40.752873 18984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "a439e054f8c547479e8c9a4baa1dc468"
format_stamp: "Formatted at 2026-08-09 12:06:36 on dist-test-slave-sb2z"
server_key: "beadb45b0c5ca7b515bbcec295a9ccbc"
server_key_iv: "576eee4f79ee819a3870430f41358871"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:40.753809 18984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:40.777689 18984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:40.781251 18984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:40.781495 18984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:40.800498 18984 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:45935
I20260809 12:06:40.800544 19084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:45935 every 8 connection(s)
I20260809 12:06:40.801990 18984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:06:40.806547 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 18984
I20260809 12:06:40.807397 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:48319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:42545
--local_ip_for_outbound_sockets=127.0.147.65
--tserver_master_addrs=127.0.147.126:45935
--webserver_port=41915
--webserver_interface=127.0.147.65
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.147.84:48319/kms with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:06:40.808746 19085 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:06:40.811970 19085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Bootstrap starting.
I20260809 12:06:40.830549 19085 log.cc:824] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:40.839308 19085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:06:40.839910 19085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Bootstrap complete.
I20260809 12:06:40.848994 19085 raft_consensus.cc:348] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } }
I20260809 12:06:40.849905 19085 raft_consensus.cc:729] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a439e054f8c547479e8c9a4baa1dc468, State: Initialized, Role: FOLLOWER
I20260809 12:06:40.850616 19085 consensus_queue.cc:260] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } }
I20260809 12:06:40.850847 19085 raft_consensus.cc:388] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:06:40.850992 19085 raft_consensus.cc:482] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:06:40.851181 19085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:06:40.854997 19085 raft_consensus.cc:504] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } }
I20260809 12:06:40.855508 19085 leader_election.cc:302] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a439e054f8c547479e8c9a4baa1dc468; no voters:
I20260809 12:06:40.856194 19085 leader_election.cc:288] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:06:40.856338 19092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:06:40.858235 19092 raft_consensus.cc:686] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [term 2 LEADER]: Becoming Leader. State: Replica: a439e054f8c547479e8c9a4baa1dc468, State: Running, Role: LEADER
I20260809 12:06:40.858887 19092 consensus_queue.cc:237] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } }
I20260809 12:06:40.859398 19085 sys_catalog.cc:564] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:06:40.865288 19096 sys_catalog.cc:455] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a439e054f8c547479e8c9a4baa1dc468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } } }
I20260809 12:06:40.865998 19096 sys_catalog.cc:458] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:40.866606 19097 sys_catalog.cc:455] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a439e054f8c547479e8c9a4baa1dc468. Latest consensus state: current_term: 2 leader_uuid: "a439e054f8c547479e8c9a4baa1dc468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439e054f8c547479e8c9a4baa1dc468" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45935 } } }
I20260809 12:06:40.866981 19097 sys_catalog.cc:458] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468 [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:40.873643 19107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:06:40.880546 19107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:06:40.881353 19085 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 12:06:40.882138 19107 catalog_manager.cc:1259] Loaded cluster ID: d6845fd86f7d414a80265d292a2abfa5
I20260809 12:06:40.882296 19107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:06:40.887390 19107 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:06:40.889340 19107 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Loaded TSK: 0
I20260809 12:06:40.891340 19107 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:06:41.255926 19089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:41.256244 19089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:41.256491 19089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:41.268309 19089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:41.268520 19089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:41.268764 19089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:06:41.286377 19089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.147.84:48319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:42545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=41915
--tserver_master_addrs=127.0.147.126:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:41.288460 19089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:41.290277 19089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:41.322695 19144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:41.324465 19143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:41.327193 19089 server_base.cc:1034] running on GCE node
W20260809 12:06:41.327489 19146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:41.329819 19089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:41.330998 19089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:41.332276 19089 hybrid_clock.cc:648] HybridClock initialized: now 1786277201332187 us; error 68 us; skew 500 ppm
I20260809 12:06:41.332716 19089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:41 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:41.337215 19089 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:06:41.349270 19089 webserver.cc:460] Webserver started at http://127.0.147.65:41915/ using document root <none> and password file <none>
I20260809 12:06:41.350063 19089 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:41.350252 19089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:06:41 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:41.377313 19089 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.003s
I20260809 12:06:41.380486 19159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:41.382644 19089 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 12:06:41.382850 19089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "24f9e917b28346afa97640ecb324e517"
format_stamp: "Formatted at 2026-08-09 12:06:38 on dist-test-slave-sb2z"
server_key: "ff6bbd791d34da47ab690840076c5d33"
server_key_iv: "24884b714f9aa6127515da4d92a7a218"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:41.384004 19089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:41.424000 19089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:41.428378 19089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:41.428719 19089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:41.429840 19089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:41.431381 19089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:41.431506 19089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:41.431667 19089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:41.431775 19089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:41.497684 19089 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:42545
I20260809 12:06:41.497727 19348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:42545 every 8 connection(s)
I20260809 12:06:41.499696 19089 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:06:41.506804 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 19089
I20260809 12:06:41.507929 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:48319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:36395
--local_ip_for_outbound_sockets=127.0.147.66
--tserver_master_addrs=127.0.147.126:45935
--webserver_port=36141
--webserver_interface=127.0.147.66
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.147.84:48319/kms with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:06:41 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:41.580534 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:41.503753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45935 (local address 127.0.147.65:34827)
0809 12:06:41.543006 (+ 39253us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:41.543093 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:41.546422 (+ 3329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:41.557024 (+ 10602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:41.557033 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:41.557766 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:41.558770 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:41.558801 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:41.561580 (+ 2779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:41.561588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:41.563874 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:41.563902 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:41.565921 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:41.565930 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:41.566239 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:41.567032 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:41.567067 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:41.570156 (+ 3089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:41.574949 (+ 4793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:41.574961 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:41.574978 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:41.575634 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:41.576302 (+ 668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:41.576307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:41.576311 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:41.576534 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:41.577666 (+ 1132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:41.577674 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:41.578112 (+ 438us) client_negotiation.cc:769] Sending connection context
0809 12:06:41.578445 (+ 333us) client_negotiation.cc:240] Negotiation successful
0809 12:06:41.579056 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38813,"thread_start_us":50924,"threads_started":1}
I20260809 12:06:41.582960 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:41.505161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:34827 (local address 127.0.147.126:45935)
0809 12:06:41.541826 (+ 36665us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:41.541858 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:41.547084 (+ 5226us) server_negotiation.cc:407] Connection header received
0809 12:06:41.547727 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:41.547752 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:41.548386 (+ 634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:41.548900 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:41.559018 (+ 10118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:41.561368 (+ 2350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:41.564101 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:41.565297 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:41.565503 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:41.570500 (+ 4997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:41.570512 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:41.570532 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:41.570592 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:41.574728 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:41.575826 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:41.575835 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:41.575844 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:41.576158 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:41.576664 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:41.576669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:41.576672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:41.577387 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:41.577573 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:41.581289 (+ 3716us) server_negotiation.cc:299] Negotiation successful
0809 12:06:41.581727 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36214,"thread_start_us":36585,"threads_started":1}
I20260809 12:06:41.587343 19349 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45935
I20260809 12:06:41.587721 19349 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:41.588660 19349 heartbeater.cc:511] Master 127.0.147.126:45935 requested a full tablet report, sending...
W20260809 12:06:41.592340 19349 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:45935 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 12:06:41.592592 19349 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:41.593333 19349 heartbeater.cc:511] Master 127.0.147.126:45935 requested a full tablet report, sending...
I20260809 12:06:41.594676 19349 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:41.595036 19349 heartbeater.cc:511] Master 127.0.147.126:45935 requested a full tablet report, sending...
W20260809 12:06:41.596232 19349 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:06:41.948566 19354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:41.948884 19354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:41.949054 19354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:41.956519 19354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:41.956671 19354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:41.956871 19354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:06:41.968186 19354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.147.84:48319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:36395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=36141
--tserver_master_addrs=127.0.147.126:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:41.969518 19354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:41.970845 19354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:41.986968 19376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:41.987965 19375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:41.990003 19378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:41.990861 19354 server_base.cc:1034] running on GCE node
I20260809 12:06:41.991686 19354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:41.995515 19354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:41.996801 19354 hybrid_clock.cc:648] HybridClock initialized: now 1786277201996695 us; error 82 us; skew 500 ppm
I20260809 12:06:41.997233 19354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:41 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:42.003846 19354 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:06:42.005610 19354 webserver.cc:460] Webserver started at http://127.0.147.66:36141/ using document root <none> and password file <none>
I20260809 12:06:42.006373 19354 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:42.006554 19354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:06:42 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:42.031873 19354 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:06:42.035827 19392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:42.037878 19354 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:06:42.038092 19354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "40b39750cb1e4fee9468127cc1c824c0"
format_stamp: "Formatted at 2026-08-09 12:06:39 on dist-test-slave-sb2z"
server_key: "eb82dc4ef176b3c8a1d92d253591413b"
server_key_iv: "bf97fda5c3610b160c6632b9e544ed78"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:42.039806 19354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:42.064419 19354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:42.069795 19354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:42.070205 19354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:42.084688 19354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:42.097321 19354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:42.097528 19354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 12:06:42.097707 19354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:42.097856 19354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:42.160454 19354 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:36395
I20260809 12:06:42.160537 19571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:36395 every 8 connection(s)
I20260809 12:06:42.162551 19354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:06:42.172483 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 19354
I20260809 12:06:42.173324 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.147.84:48319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:34461
--local_ip_for_outbound_sockets=127.0.147.67
--tserver_master_addrs=127.0.147.126:45935
--webserver_port=36395
--webserver_interface=127.0.147.67
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.147.84:48319/kms with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:06:42 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:42.200445 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:42.165393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:39601 (local address 127.0.147.126:45935)
0809 12:06:42.180592 (+ 15199us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:42.180599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:42.180624 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:06:42.180695 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:42.180701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:42.180798 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:42.180990 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:42.183498 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.184713 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.187252 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.187807 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.187911 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:42.192293 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:42.192300 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:42.192305 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:42.192347 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:42.196178 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:42.197155 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:42.197161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:42.197166 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:42.197320 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:42.197899 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:42.197903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:42.197907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:42.198275 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:42.198421 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:42.199388 (+ 967us) server_negotiation.cc:299] Negotiation successful
0809 12:06:42.199775 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15051,"thread_start_us":15751,"threads_started":1}
I20260809 12:06:42.201261 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:42.165964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45935 (local address 127.0.147.66:39601)
0809 12:06:42.173135 (+ 7171us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:42.173192 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:42.176581 (+ 3389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:42.181294 (+ 4713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:42.181310 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:42.182366 (+ 1056us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:42.183242 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:42.183271 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.184929 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.184934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:42.187063 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:42.187104 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.188068 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.188078 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:42.188388 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:42.189201 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:42.189229 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:42.192017 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:42.196347 (+ 4330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:42.196356 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:42.196374 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:42.196991 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:42.197466 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:42.197471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:42.197475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:42.197753 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:42.198520 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:42.198529 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:42.198923 (+ 394us) client_negotiation.cc:769] Sending connection context
0809 12:06:42.199265 (+ 342us) client_negotiation.cc:240] Negotiation successful
0809 12:06:42.199856 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6824,"thread_start_us":17357,"threads_started":1}
Aug 09 12:06:42 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:42.207680 19572 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45935
I20260809 12:06:42.208079 19572 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:42.209117 19572 heartbeater.cc:511] Master 127.0.147.126:45935 requested a full tablet report, sending...
I20260809 12:06:42.211275 19028 ts_manager.cc:194] Registered new tserver with Master: 40b39750cb1e4fee9468127cc1c824c0 (127.0.147.66:36395)
I20260809 12:06:42.213482 19028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:39601
I20260809 12:06:42.596359 19349 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:42.596793 19349 heartbeater.cc:511] Master 127.0.147.126:45935 requested a full tablet report, sending...
I20260809 12:06:42.598137 19028 ts_manager.cc:194] Registered new tserver with Master: 24f9e917b28346afa97640ecb324e517 (127.0.147.65:42545)
I20260809 12:06:42.599323 19028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:34827
Aug 09 12:06:42 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
W20260809 12:06:42.629441 19578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:42.629879 19578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:42.630069 19578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:42.641283 19578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:42.641420 19578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:42.641659 19578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:06:42.656844 19578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.147.84:48319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:34461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=36395
--tserver_master_addrs=127.0.147.126:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:42.658733 19578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:42.660463 19578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:42.677964 19589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:42.677992 19590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:42.682628 19578 server_base.cc:1034] running on GCE node
W20260809 12:06:42.683290 19592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:42.684113 19578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:42.685307 19578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:42.686591 19578 hybrid_clock.cc:648] HybridClock initialized: now 1786277202686520 us; error 53 us; skew 500 ppm
I20260809 12:06:42.686960 19578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:42 dist-test-slave-sb2z krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:42.691190 19578 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:06:42.696954 19578 webserver.cc:460] Webserver started at http://127.0.147.67:36395/ using document root <none> and password file <none>
I20260809 12:06:42.697851 19578 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:42.697999 19578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:06:42 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:42.723488 19578 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:06:42.727121 19605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:42.728919 19578 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260809 12:06:42.729106 19578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "abafa36ef7c148f8b7cef4de1bfee866"
format_stamp: "Formatted at 2026-08-09 12:06:39 on dist-test-slave-sb2z"
server_key: "3da3672505aefac93c902af78da94b3d"
server_key_iv: "828bbd1d08919e1b55894d502dffdc78"
server_key_version: "kuduclusterkey@0"
I20260809 12:06:42.730525 19578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:42.761420 19578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:42.766711 19578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:42.767060 19578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:42.768361 19578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:42.770195 19578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:42.770318 19578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:42.770454 19578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:42.770539 19578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:42.852756 19578 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:34461
I20260809 12:06:42.852802 19776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:34461 every 8 connection(s)
I20260809 12:06:42.854318 19578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:06:42.860466 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 19578
I20260809 12:06:42.891889 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:42.862104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54802 (local address 127.0.147.126:45935)
0809 12:06:42.873845 (+ 11741us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:42.873851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:42.873868 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:06:42.873938 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:42.873943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:42.874052 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:42.874159 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:42.876640 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.877721 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.879682 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.880306 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.880419 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:42.883927 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:42.883934 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:42.883938 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:42.884035 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:42.887620 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:42.888685 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:42.888691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:42.888695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:42.888850 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:42.889324 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:42.889330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:42.889334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:42.889704 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:42.889984 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:42.891230 (+ 1246us) server_negotiation.cc:299] Negotiation successful
0809 12:06:42.891462 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11606,"thread_start_us":281,"threads_started":1}
Aug 09 12:06:42 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:42.909665 19785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:42.859221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:41609 (local address 127.0.147.126:45935)
0809 12:06:42.862657 (+ 3436us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:42.862666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:42.873354 (+ 10688us) server_negotiation.cc:407] Connection header received
0809 12:06:42.877601 (+ 4247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:42.877607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:42.877717 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:42.877803 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:42.882269 (+ 4466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.883440 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.895209 (+ 11769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.895894 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.896040 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:42.901272 (+ 5232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:42.901279 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:42.901285 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:42.901325 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:42.904777 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:42.906163 (+ 1386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:42.906170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:42.906177 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:42.906641 (+ 464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:42.907271 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:42.907276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:42.907280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:42.907802 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:42.907937 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:42.909008 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0809 12:06:42.909271 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3262,"thread_start_us":14477,"threads_started":1}
I20260809 12:06:42.912305 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:42.858426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45935 (local address 127.0.147.67:41609)
0809 12:06:42.872992 (+ 14566us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:42.873065 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:42.876342 (+ 3277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:42.878885 (+ 2543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:42.878898 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:42.881127 (+ 2229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:42.882027 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:42.882057 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.886262 (+ 4205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.886267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:42.893700 (+ 7433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:42.893729 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.896151 (+ 2422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.896160 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:42.896472 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:42.898337 (+ 1865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:42.898371 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:42.900986 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:42.904944 (+ 3958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:42.904954 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:42.904973 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:42.905985 (+ 1012us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:42.906812 (+ 827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:42.906817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:42.906820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:42.907108 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:42.908064 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:42.908072 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:42.908531 (+ 459us) client_negotiation.cc:769] Sending connection context
0809 12:06:42.910226 (+ 1695us) client_negotiation.cc:240] Negotiation successful
0809 12:06:42.910820 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14254,"thread_start_us":14031,"threads_started":1}
I20260809 12:06:42.914131 19777 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45935
I20260809 12:06:42.914453 19777 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:42.915328 19777 heartbeater.cc:511] Master 127.0.147.126:45935 requested a full tablet report, sending...
I20260809 12:06:42.918561 19028 ts_manager.cc:194] Registered new tserver with Master: abafa36ef7c148f8b7cef4de1bfee866 (127.0.147.67:34461)
I20260809 12:06:42.919793 19028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:41609
I20260809 12:06:42.920672 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:06:42 dist-test-slave-sb2z krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for HTTP/127.0.147.84@KRBTEST.COM
I20260809 12:06:42.961802 19785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:42.938339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.0.147.126:45935)
0809 12:06:42.938529 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:42.938535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:42.943910 (+ 5375us) server_negotiation.cc:407] Connection header received
0809 12:06:42.944125 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:42.944130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:42.944229 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:42.944342 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:42.946458 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.947627 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.949761 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:42.950467 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:42.950583 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:42.954542 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:42.954548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:42.954553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:42.954596 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:42.958222 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:42.959135 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:42.959143 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:42.959146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:42.959304 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:42.959848 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:42.959853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:42.959857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:42.960276 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:42.960497 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:42.961177 (+ 680us) server_negotiation.cc:299] Negotiation successful
0809 12:06:42.961428 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:06:42.969859 19028 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:06:42.980872 19028 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:06:43.084582 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.031973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36395 (local address 127.0.0.1:35022)
0809 12:06:43.032659 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:43.032709 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:43.032881 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:43.073120 (+ 40239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:43.073128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:43.073178 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:43.073737 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.073752 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.078126 (+ 4374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.078132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.080314 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.080329 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.083739 (+ 3410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.083744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.083931 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.083964 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:06:43.084040 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 12:06:43.084189 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":4,"thread_start_us":710,"threads_started":1}
I20260809 12:06:43.085155 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.032516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.0.147.66:36395)
0809 12:06:43.072267 (+ 39751us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:43.072276 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:43.072311 (+ 35us) server_negotiation.cc:407] Connection header received
0809 12:06:43.072449 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:43.072463 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:43.072825 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:43.073114 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:43.074428 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.076387 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.080569 (+ 4182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.083463 (+ 2894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.084312 (+ 849us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.084521 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:43.084605 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 12:06:43.084786 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39562,"thread_start_us":41644,"threads_started":1}
I20260809 12:06:43.086009 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.044375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40510 (local address 127.0.147.65:42545)
0809 12:06:43.069034 (+ 24659us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:43.069066 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:43.074148 (+ 5082us) server_negotiation.cc:407] Connection header received
0809 12:06:43.074270 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:43.074279 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:43.074536 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:43.074979 (+ 443us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:43.075859 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.077428 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.081030 (+ 3602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.082466 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.082592 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.082875 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:43.085525 (+ 2650us) server_negotiation.cc:299] Negotiation successful
0809 12:06:43.085705 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24393,"thread_start_us":24718,"threads_started":1}
I20260809 12:06:43.084182 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.043795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:42545 (local address 127.0.0.1:40510)
0809 12:06:43.073870 (+ 30075us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:43.073890 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:43.074001 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:43.075263 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:43.075267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:43.075293 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:43.075637 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.075649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.077634 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.077639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.080856 (+ 3217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.080865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.083044 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.083052 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.083267 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.083327 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 12:06:43.083474 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 12:06:43.083733 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29934,"mutex_wait_us":133,"thread_start_us":30138,"threads_started":1}
I20260809 12:06:43.089501 19473 tablet_service.cc:1467] Processing CreateTablet for tablet 89426457936b4020832a9df9d37a7c66 (DEFAULT_TABLE table=test-table [id=f71259cb1c94410a95bedc4d5a13864f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:06:43.091101 19473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89426457936b4020832a9df9d37a7c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:43.094743 19242 tablet_service.cc:1467] Processing CreateTablet for tablet 89426457936b4020832a9df9d37a7c66 (DEFAULT_TABLE table=test-table [id=f71259cb1c94410a95bedc4d5a13864f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:06:43.095386 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.035708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:34461 (local address 127.0.0.1:37860)
0809 12:06:43.071365 (+ 35657us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:43.071387 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:43.071560 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:43.080239 (+ 8679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:43.080245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:43.080277 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:43.080690 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.080701 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.089058 (+ 8357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.089065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.091041 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.091058 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.094070 (+ 3012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.094077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.094266 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.094304 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:06:43.094375 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 12:06:43.094515 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35471,"mutex_wait_us":18,"thread_start_us":35958,"threads_started":1}
I20260809 12:06:43.096287 19242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89426457936b4020832a9df9d37a7c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:43.096478 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.035982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37860 (local address 127.0.147.67:34461)
0809 12:06:43.077089 (+ 41107us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:43.077100 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:43.077131 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:06:43.077215 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:43.077222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:43.077431 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:43.077685 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:43.086507 (+ 8822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.088200 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.092241 (+ 4041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.093794 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.095633 (+ 1839us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.095879 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:43.095967 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 12:06:43.096128 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40930,"thread_start_us":40763,"threads_started":1}
I20260809 12:06:43.100167 19678 tablet_service.cc:1467] Processing CreateTablet for tablet 89426457936b4020832a9df9d37a7c66 (DEFAULT_TABLE table=test-table [id=f71259cb1c94410a95bedc4d5a13864f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:06:43.101884 19678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89426457936b4020832a9df9d37a7c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:43.124233 19811 tablet_bootstrap.cc:492] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: Bootstrap starting.
I20260809 12:06:43.136574 19811 tablet_bootstrap.cc:654] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:43.138082 19811 log.cc:824] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:43.142108 19812 tablet_bootstrap.cc:492] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: Bootstrap starting.
I20260809 12:06:43.145605 19811 tablet_bootstrap.cc:492] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: No bootstrap required, opened a new log
I20260809 12:06:43.145974 19811 ts_tablet_manager.cc:1397] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.005s
I20260809 12:06:43.146170 19814 tablet_bootstrap.cc:492] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: Bootstrap starting.
I20260809 12:06:43.150169 19814 tablet_bootstrap.cc:654] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:43.150272 19812 tablet_bootstrap.cc:654] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:43.151667 19814 log.cc:824] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:43.152594 19812 log.cc:824] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:43.164347 19812 tablet_bootstrap.cc:492] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: No bootstrap required, opened a new log
I20260809 12:06:43.164716 19812 ts_tablet_manager.cc:1397] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.006s
I20260809 12:06:43.165686 19811 raft_consensus.cc:348] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } }
I20260809 12:06:43.171739 19811 raft_consensus.cc:374] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:43.171921 19811 raft_consensus.cc:729] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f9e917b28346afa97640ecb324e517, State: Initialized, Role: FOLLOWER
I20260809 12:06:43.172715 19811 consensus_queue.cc:260] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } }
I20260809 12:06:43.174010 19814 tablet_bootstrap.cc:492] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: No bootstrap required, opened a new log
I20260809 12:06:43.174459 19814 ts_tablet_manager.cc:1397] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.016s
W20260809 12:06:43.174989 19566 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.0.147.66:36395
I20260809 12:06:43.178372 19349 heartbeater.cc:503] Master 127.0.147.126:45935 was elected leader, sending a full tablet report...
I20260809 12:06:43.180729 19811 ts_tablet_manager.cc:1428] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: Time spent starting tablet: real 0.035s user 0.012s sys 0.001s
I20260809 12:06:43.185205 19814 raft_consensus.cc:348] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } }
I20260809 12:06:43.185761 19814 raft_consensus.cc:374] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:43.185932 19814 raft_consensus.cc:729] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b39750cb1e4fee9468127cc1c824c0, State: Initialized, Role: FOLLOWER
I20260809 12:06:43.186800 19814 consensus_queue.cc:260] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } }
I20260809 12:06:43.190060 19812 raft_consensus.cc:348] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } }
I20260809 12:06:43.190680 19812 raft_consensus.cc:374] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:43.190831 19812 raft_consensus.cc:729] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abafa36ef7c148f8b7cef4de1bfee866, State: Initialized, Role: FOLLOWER
I20260809 12:06:43.191656 19812 consensus_queue.cc:260] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } }
I20260809 12:06:43.199576 19777 heartbeater.cc:503] Master 127.0.147.126:45935 was elected leader, sending a full tablet report...
I20260809 12:06:43.199940 19812 ts_tablet_manager.cc:1428] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: Time spent starting tablet: real 0.035s user 0.019s sys 0.002s
I20260809 12:06:43.204803 19572 heartbeater.cc:503] Master 127.0.147.126:45935 was elected leader, sending a full tablet report...
I20260809 12:06:43.205576 19814 ts_tablet_manager.cc:1428] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: Time spent starting tablet: real 0.031s user 0.016s sys 0.012s
I20260809 12:06:43.253120 19818 raft_consensus.cc:482] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:06:43.253458 19818 raft_consensus.cc:504] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } }
I20260809 12:06:43.255295 19818 leader_election.cc:288] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40b39750cb1e4fee9468127cc1c824c0 (127.0.147.66:36395), abafa36ef7c148f8b7cef4de1bfee866 (127.0.147.67:34461)
I20260809 12:06:43.263013 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.256480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:42957 (local address 127.0.147.67:34461)
0809 12:06:43.256662 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:43.256670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:43.257500 (+ 830us) server_negotiation.cc:407] Connection header received
0809 12:06:43.257722 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:43.257725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:43.257807 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:43.257897 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:43.258541 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.259225 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.261259 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.262019 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.262205 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.262285 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:43.262612 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 12:06:43.262730 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:06:43.263087 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.255973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:43371 (local address 127.0.147.66:36395)
0809 12:06:43.256235 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:43.256244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:43.257147 (+ 903us) server_negotiation.cc:407] Connection header received
0809 12:06:43.257410 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:43.257415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:43.257489 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:43.257593 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:43.258180 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.259022 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.261248 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.262031 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.262216 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.262304 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:43.262659 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 12:06:43.262801 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:06:43.263173 19834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.255865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:34461 (local address 127.0.147.65:42957)
0809 12:06:43.257186 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:43.257206 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:43.257565 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:43.257912 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:43.257917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:43.257944 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:43.258360 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.258375 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.259367 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.259372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.261104 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.261116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.262269 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.262276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.262455 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.262477 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 12:06:43.262584 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 12:06:43.262703 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1180,"mutex_wait_us":54,"thread_start_us":563,"threads_started":1}
I20260809 12:06:43.263197 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.255678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36395 (local address 127.0.147.65:43371)
0809 12:06:43.256939 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:43.256962 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:43.257215 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:43.257615 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:43.257619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:43.257646 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:43.258026 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.258039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.259161 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.259167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.261098 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:43.261110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.262322 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.262330 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:43.262502 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.262524 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 12:06:43.262632 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 12:06:43.262750 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":114,"thread_start_us":1141,"threads_started":1}
I20260809 12:06:43.264433 19709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89426457936b4020832a9df9d37a7c66" candidate_uuid: "24f9e917b28346afa97640ecb324e517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abafa36ef7c148f8b7cef4de1bfee866" is_pre_election: true
I20260809 12:06:43.264549 19504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89426457936b4020832a9df9d37a7c66" candidate_uuid: "24f9e917b28346afa97640ecb324e517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b39750cb1e4fee9468127cc1c824c0" is_pre_election: true
I20260809 12:06:43.264971 19709 raft_consensus.cc:2454] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f9e917b28346afa97640ecb324e517 in term 0.
I20260809 12:06:43.265117 19504 raft_consensus.cc:2454] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f9e917b28346afa97640ecb324e517 in term 0.
I20260809 12:06:43.265832 19164 leader_election.cc:302] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f9e917b28346afa97640ecb324e517, abafa36ef7c148f8b7cef4de1bfee866; no voters:
I20260809 12:06:43.266234 19818 raft_consensus.cc:2781] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:06:43.266384 19818 raft_consensus.cc:482] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:06:43.266503 19818 raft_consensus.cc:3037] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:43.268795 19818 raft_consensus.cc:504] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } }
I20260809 12:06:43.269488 19818 leader_election.cc:288] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [CANDIDATE]: Term 1 election: Requested vote from peers 40b39750cb1e4fee9468127cc1c824c0 (127.0.147.66:36395), abafa36ef7c148f8b7cef4de1bfee866 (127.0.147.67:34461)
I20260809 12:06:43.269932 19504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89426457936b4020832a9df9d37a7c66" candidate_uuid: "24f9e917b28346afa97640ecb324e517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b39750cb1e4fee9468127cc1c824c0"
I20260809 12:06:43.269979 19709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89426457936b4020832a9df9d37a7c66" candidate_uuid: "24f9e917b28346afa97640ecb324e517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abafa36ef7c148f8b7cef4de1bfee866"
I20260809 12:06:43.270146 19504 raft_consensus.cc:3037] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:43.270164 19709 raft_consensus.cc:3037] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:43.272370 19504 raft_consensus.cc:2454] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f9e917b28346afa97640ecb324e517 in term 1.
I20260809 12:06:43.272652 19709 raft_consensus.cc:2454] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f9e917b28346afa97640ecb324e517 in term 1.
I20260809 12:06:43.272856 19163 leader_election.cc:302] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f9e917b28346afa97640ecb324e517, 40b39750cb1e4fee9468127cc1c824c0; no voters:
I20260809 12:06:43.273170 19818 raft_consensus.cc:2781] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:06:43.273736 19818 raft_consensus.cc:686] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 1 LEADER]: Becoming Leader. State: Replica: 24f9e917b28346afa97640ecb324e517, State: Running, Role: LEADER
I20260809 12:06:43.274299 19818 consensus_queue.cc:237] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } }
I20260809 12:06:43.279675 19027 catalog_manager.cc:5495] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24f9e917b28346afa97640ecb324e517 (127.0.147.65). New cstate: current_term: 1 leader_uuid: "24f9e917b28346afa97640ecb324e517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f9e917b28346afa97640ecb324e517" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 42545 } health_report { overall_health: HEALTHY } } }
W20260809 12:06:43.308291 19351 tablet.cc:2369] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:06:43.338654 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.333541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.0.147.65:42545)
0809 12:06:43.333758 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:43.333763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:43.333782 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:06:43.333868 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:43.333872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:43.333958 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:43.334108 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:06:43.334431 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.335177 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.336636 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.337027 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.337120 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.337646 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:06:43.338155 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:06:43.338270 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:43.338378 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 12:06:43.338444 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:06:43.347115 19709 raft_consensus.cc:1261] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [term 1 FOLLOWER]: Refusing update from remote peer 24f9e917b28346afa97640ecb324e517: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:06:43.347115 19504 raft_consensus.cc:1261] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [term 1 FOLLOWER]: Refusing update from remote peer 24f9e917b28346afa97640ecb324e517: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:06:43.348171 19837 consensus_queue.cc:1035] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b39750cb1e4fee9468127cc1c824c0" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:06:43.348587 19818 consensus_queue.cc:1035] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abafa36ef7c148f8b7cef4de1bfee866" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:06:43.379092 19027 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:06:43.379391 19027 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:06:43.383117 19027 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a439e054f8c547479e8c9a4baa1dc468: Sending DeleteTablet for 3 replicas of tablet 89426457936b4020832a9df9d37a7c66
I20260809 12:06:43.384411 19473 tablet_service.cc:1514] Processing DeleteTablet for tablet 89426457936b4020832a9df9d37a7c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:43 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:35022
I20260809 12:06:43.384770 19242 tablet_service.cc:1514] Processing DeleteTablet for tablet 89426457936b4020832a9df9d37a7c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:43 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:40510
I20260809 12:06:43.385334 19862 tablet_replica.cc:330] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: stopping tablet replica
I20260809 12:06:43.385723 19862 raft_consensus.cc:2229] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:06:43.385711 19678 tablet_service.cc:1514] Processing DeleteTablet for tablet 89426457936b4020832a9df9d37a7c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:43 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:37860
I20260809 12:06:43.386216 19862 raft_consensus.cc:2258] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:06:43.388546 19862 ts_tablet_manager.cc:1905] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:06:43.388669 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 19089
I20260809 12:06:43.397010 19862 ts_tablet_manager.cc:1918] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:06:43.397315 19862 log.cc:1196] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/89426457936b4020832a9df9d37a7c66
I20260809 12:06:43.398339 19865 tablet_replica.cc:330] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: stopping tablet replica
I20260809 12:06:43.398833 19865 raft_consensus.cc:2229] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:06:43.399577 19862 ts_tablet_manager.cc:1939] T 89426457936b4020832a9df9d37a7c66 P 40b39750cb1e4fee9468127cc1c824c0: Deleting consensus metadata
I20260809 12:06:43.401232 19865 raft_consensus.cc:2258] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:06:43.402557 19014 catalog_manager.cc:4841] TS 40b39750cb1e4fee9468127cc1c824c0 (127.0.147.66:36395): tablet 89426457936b4020832a9df9d37a7c66 (table test-table [id=f71259cb1c94410a95bedc4d5a13864f]) successfully deleted
I20260809 12:06:43.404016 19863 tablet_replica.cc:330] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: stopping tablet replica
I20260809 12:06:43.404433 19863 raft_consensus.cc:2229] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:06:43.405259 19863 raft_consensus.cc:2258] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:06:43.407158 19865 ts_tablet_manager.cc:1905] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:06:43.407418 19863 ts_tablet_manager.cc:1905] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:06:43.411950 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.390025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40530 (local address 127.0.147.65:42545)
0809 12:06:43.390215 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:43.390221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:43.390684 (+ 463us) server_negotiation.cc:407] Connection header received
0809 12:06:43.390861 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:43.390869 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:43.390949 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:43.391056 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:43.397638 (+ 6582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.398905 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.400801 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.401520 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.401661 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.404555 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:43.404569 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:43.404585 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:43.404626 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:43.408933 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:43.409641 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:43.409648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:43.409652 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:43.409819 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:43.410221 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:43.410226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:43.410229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:43.410679 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:43.411026 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:43.411295 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 12:06:43.411578 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:06:43.412911 19338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:43.416723 19865 ts_tablet_manager.cc:1918] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:06:43.416947 19865 log.cc:1196] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/89426457936b4020832a9df9d37a7c66
I20260809 12:06:43.417646 19865 ts_tablet_manager.cc:1939] T 89426457936b4020832a9df9d37a7c66 P abafa36ef7c148f8b7cef4de1bfee866: Deleting consensus metadata
I20260809 12:06:43.419457 19015 catalog_manager.cc:4841] TS abafa36ef7c148f8b7cef4de1bfee866 (127.0.147.67:34461): tablet 89426457936b4020832a9df9d37a7c66 (table test-table [id=f71259cb1c94410a95bedc4d5a13864f]) successfully deleted
I20260809 12:06:43.514099 19863 ts_tablet_manager.cc:1918] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:06:43.514354 19863 log.cc:1196] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/89426457936b4020832a9df9d37a7c66
I20260809 12:06:43.515095 19863 ts_tablet_manager.cc:1939] T 89426457936b4020832a9df9d37a7c66 P 24f9e917b28346afa97640ecb324e517: Deleting consensus metadata
I20260809 12:06:43.517140 19012 catalog_manager.cc:4841] TS 24f9e917b28346afa97640ecb324e517 (127.0.147.65:42545): tablet 89426457936b4020832a9df9d37a7c66 (table test-table [id=f71259cb1c94410a95bedc4d5a13864f]) successfully deleted
I20260809 12:06:43.523885 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 19089
I20260809 12:06:43.550383 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 19354
I20260809 12:06:43.564575 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.551175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.0.147.66:36395)
0809 12:06:43.551383 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:43.551389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:43.551491 (+ 102us) server_negotiation.cc:407] Connection header received
0809 12:06:43.551776 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:43.551788 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:43.551863 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:43.551972 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:43.553333 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.554076 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.555320 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.555865 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.556007 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.558717 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:43.558730 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:43.558746 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:43.558776 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:43.562040 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:43.562680 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:43.562688 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:43.562690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:43.562835 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:43.563158 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:43.563162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:43.563164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:43.563614 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:43.563862 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:43.564100 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 12:06:43.564268 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:06:43.565388 19564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:43.657050 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 19354
I20260809 12:06:43.685087 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 19578
I20260809 12:06:43.699785 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:43.685877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37866 (local address 127.0.147.67:34461)
0809 12:06:43.686094 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:43.686102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:43.686126 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:06:43.686352 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:43.686357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:43.686422 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:43.686535 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:43.687993 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.688744 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.690097 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:43.690501 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:43.690641 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:43.693797 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:43.693813 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:43.693830 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:43.693872 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:43.696860 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:43.697573 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:43.697589 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:43.697592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:43.697738 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:43.698140 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:43.698146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:43.698149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:43.698542 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:43.698860 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:43.699192 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 12:06:43.699418 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:06:43.700522 19772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:43.789137 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 19578
I20260809 12:06:43.813448 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 18984
I20260809 12:06:43.814718 19079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:43.929806 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 18984
I20260809 12:06:43.962253 589 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 12:06:44.307181 589 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 12:06:44.307359 589 mini_ranger.cc:66] Stopping Ranger...
I20260809 12:06:44.662210 589 mini_ranger.cc:68] Stopped Ranger
2026-08-09 12:06:44.662 UTC [17598] LOG: received smart shutdown request
2026-08-09 12:06:44.665 UTC [17598] LOG: background worker "logical replication launcher" (PID 17608) exited with exit code 1
2026-08-09 12:06:44.665 UTC [17602] LOG: shutting down
2026-08-09 12:06:44.806 UTC [17598] LOG: database system is shut down
2026-08-09T12:06:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53924 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:06:45 dist-test-slave-sb2z krb5kdc[19887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:06:45 dist-test-slave-sb2z krb5kdc[19887](info): set up 2 sockets
Aug 09 12:06:45 dist-test-slave-sb2z krb5kdc[19887](info): commencing operation
krb5kdc: starting...
W20260809 12:06:46.266566 589 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:06:46 dist-test-slave-sb2z krb5kdc[19887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:06:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:06:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:06:46.561216 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39131
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:39131,127.0.147.125:45421,127.0.147.124:35457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:06:46.912948 19903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:46.913287 19903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:46.913456 19903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:46.920468 19903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:06:46.920614 19903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:46.920723 19903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:46.920835 19903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:06:46.920953 19903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:06:46.930398 19903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:39131,127.0.147.125:45421,127.0.147.124:35457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:46.931731 19903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:46.932986 19903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:46.944664 19909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:46.944705 19912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:46.944623 19910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:46.945717 19903 server_base.cc:1034] running on GCE node
I20260809 12:06:46.946542 19903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:46.947906 19903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:46.949147 19903 hybrid_clock.cc:648] HybridClock initialized: now 1786277206949116 us; error 41 us; skew 500 ppm
I20260809 12:06:46.949601 19903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:46 dist-test-slave-sb2z krb5kdc[19887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:46.952828 19903 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:06:46.954221 19903 webserver.cc:460] Webserver started at http://127.0.147.126:42803/ using document root <none> and password file <none>
I20260809 12:06:46.954891 19903 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:46.955045 19903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:46.955349 19903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:06:46.958302 19903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "c2a9fb0550704c9da616b59f629eb44a"
format_stamp: "Formatted at 2026-08-09 12:06:46 on dist-test-slave-sb2z"
server_key: "2511be26dd58720907cce38d1c39cc3e"
server_key_iv: "0f3f51a8cc494374859aa9d910542ab9"
server_key_version: "encryptionkey@0"
I20260809 12:06:46.959102 19903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "c2a9fb0550704c9da616b59f629eb44a"
format_stamp: "Formatted at 2026-08-09 12:06:46 on dist-test-slave-sb2z"
server_key: "2511be26dd58720907cce38d1c39cc3e"
server_key_iv: "0f3f51a8cc494374859aa9d910542ab9"
server_key_version: "encryptionkey@0"
I20260809 12:06:46.964365 19903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 12:06:46.967830 19924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:46.969108 19903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:06:46.969300 19903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "c2a9fb0550704c9da616b59f629eb44a"
format_stamp: "Formatted at 2026-08-09 12:06:46 on dist-test-slave-sb2z"
server_key: "2511be26dd58720907cce38d1c39cc3e"
server_key_iv: "0f3f51a8cc494374859aa9d910542ab9"
server_key_version: "encryptionkey@0"
I20260809 12:06:46.969504 19903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:46.987484 19903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:46.991346 19903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:46.991696 19903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:47.010123 19903 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:39131
I20260809 12:06:47.010162 19996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:39131 every 8 connection(s)
I20260809 12:06:47.011430 19903 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:06:47.012689 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 19903
I20260809 12:06:47.012931 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:06:47.013592 589 external_mini_cluster.cc:1428] Setting key 0f3b940cf77258232de6c9a73613e614
I20260809 12:06:47.015395 19997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:47.021915 19997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:06:47 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.147.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.125@KRBTEST.COM" created.
I20260809 12:06:47.049805 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.025164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.0.147.126:39131)
0809 12:06:47.025996 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:47.026011 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:47.026463 (+ 452us) server_negotiation.cc:407] Connection header received
0809 12:06:47.029156 (+ 2693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:47.029162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:47.029378 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:47.029555 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:47.031246 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.032665 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.034935 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.035471 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.035715 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:47.042309 (+ 6594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:47.042317 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:47.042329 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:47.042362 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:47.045306 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:47.045997 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:47.046005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:47.046009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:47.046178 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:47.046592 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:47.046600 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:47.046603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:47.047088 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:47.047295 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:47.047904 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 12:06:47.048331 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":592,"thread_start_us":403,"threads_started":1}
I20260809 12:06:47.049824 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.025177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39131 (local address 127.0.0.1:55910)
0809 12:06:47.026312 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.026345 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:47.028766 (+ 2421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:47.029627 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:47.029641 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:47.030518 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:47.031018 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:47.031045 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.032822 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.032830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:47.034783 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:47.034801 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.035803 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.035813 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:47.036047 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:47.039225 (+ 3178us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:47.039260 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:47.042075 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:47.045439 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:47.045452 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:47.045464 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:47.045864 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:47.046297 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:47.046302 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:47.046305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:47.046484 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:47.047244 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:47.047252 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:47.047477 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 12:06:47.047803 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 12:06:47.048443 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"mutex_wait_us":174,"thread_start_us":762,"threads_started":1}
I20260809 12:06:47.052788 19997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:06:47.053949 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.053568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38188)
0809 12:06:47.053736 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.053845 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 12:06:47.054347 19930 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
I20260809 12:06:47.055984 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.055609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38192)
0809 12:06:47.055779 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.055849 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:06:47.056286 19997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:45421: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab.
Entry for principal kudu/127.0.147.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab.
I20260809 12:06:47.080279 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.125:45421
--webserver_interface=127.0.147.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab
--principal=kudu/127.0.147.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:39131,127.0.147.125:45421,127.0.147.124:35457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:06:47.105669 19997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } attempt: 1
I20260809 12:06:47.106995 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.106528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38206)
0809 12:06:47.106742 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.106854 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 12:06:47.108083 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.107761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38222)
0809 12:06:47.107893 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.107960 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 12:06:47.108327 19997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:45421: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
I20260809 12:06:47.176606 19997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } attempt: 2
I20260809 12:06:47.177891 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.177398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38228)
0809 12:06:47.177641 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.177753 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 12:06:47.178937 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.178564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38232)
0809 12:06:47.178715 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.178810 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:06:47.179133 19997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:45421: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
I20260809 12:06:47.270429 19997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } attempt: 3
I20260809 12:06:47.271924 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.271426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38240)
0809 12:06:47.271661 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.271781 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 12:06:47.273036 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.272631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38256)
0809 12:06:47.272787 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.272866 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 12:06:47.273353 19997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:45421: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
I20260809 12:06:47.416644 19997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } attempt: 4
I20260809 12:06:47.417862 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.417318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38268)
0809 12:06:47.417632 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.417718 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260809 12:06:47.418958 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.418591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38274)
0809 12:06:47.418740 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.418832 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:06:47.419220 19997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:45421: Network error: Client connection negotiation failed: client connection to 127.0.147.125:45421: connect: Connection refused (error 111)
W20260809 12:06:47.421972 20009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:47.422271 20009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:47.422438 20009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:47.429721 20009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:06:47.429847 20009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:47.429936 20009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:47.430049 20009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:06:47.430135 20009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:06:47.439860 20009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:39131,127.0.147.125:45421,127.0.147.124:35457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab
--principal=kudu/127.0.147.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.125:45421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.147.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:47.441202 20009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:47.442513 20009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:47.454221 20016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:47.454155 20018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:47.454813 20022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:47.455204 20009 server_base.cc:1034] running on GCE node
I20260809 12:06:47.456028 20009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:47.457310 20009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:47.458523 20009 hybrid_clock.cc:648] HybridClock initialized: now 1786277207458480 us; error 32 us; skew 500 ppm
I20260809 12:06:47.458881 20009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:47 dist-test-slave-sb2z krb5kdc[19887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:47.462378 20009 init.cc:377] Logged in from keytab as kudu/127.0.147.125@KRBTEST.COM (short username kudu)
I20260809 12:06:47.464005 20009 webserver.cc:460] Webserver started at http://127.0.147.125:33239/ using document root <none> and password file <none>
I20260809 12:06:47.464696 20009 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:47.464836 20009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:47.465163 20009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:06:47.468225 20009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/data/instance:
uuid: "9736a4f2c3d049b1b7aaef6cbc065e40"
format_stamp: "Formatted at 2026-08-09 12:06:47 on dist-test-slave-sb2z"
server_key: "81abe46756287bff3aa0340d1ffd622e"
server_key_iv: "2d6eae8aa27d92a46c3d9728ec69bf49"
server_key_version: "encryptionkey@0"
I20260809 12:06:47.469069 20009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/wal/instance:
uuid: "9736a4f2c3d049b1b7aaef6cbc065e40"
format_stamp: "Formatted at 2026-08-09 12:06:47 on dist-test-slave-sb2z"
server_key: "81abe46756287bff3aa0340d1ffd622e"
server_key_iv: "2d6eae8aa27d92a46c3d9728ec69bf49"
server_key_version: "encryptionkey@0"
I20260809 12:06:47.474251 20009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:06:47.477715 20040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:47.479063 20009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 12:06:47.479255 20009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/wal
uuid: "9736a4f2c3d049b1b7aaef6cbc065e40"
format_stamp: "Formatted at 2026-08-09 12:06:47 on dist-test-slave-sb2z"
server_key: "81abe46756287bff3aa0340d1ffd622e"
server_key_iv: "2d6eae8aa27d92a46c3d9728ec69bf49"
server_key_version: "encryptionkey@0"
I20260809 12:06:47.479439 20009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:47.501595 20009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:47.505653 20009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:47.505887 20009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:47.525074 20009 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.125:45421
I20260809 12:06:47.525108 20108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.125:45421 every 8 connection(s)
I20260809 12:06:47.526458 20009 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
I20260809 12:06:47.530170 20109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:47.530368 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 20009
I20260809 12:06:47.530596 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-1/wal/instance
I20260809 12:06:47.531075 589 external_mini_cluster.cc:1428] Setting key ab81ce4d7c0251d5108a1e2735d74804
I20260809 12:06:47.536823 20109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:06:47 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:47.558717 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.539482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.0.147.126:39131)
0809 12:06:47.539682 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:47.539687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:47.541160 (+ 1473us) server_negotiation.cc:407] Connection header received
0809 12:06:47.543063 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:47.543069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:47.543158 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:47.543276 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:47.544878 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.545571 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.547111 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.547591 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.547724 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:47.551210 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:47.551219 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:47.551227 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:47.551275 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:47.555426 (+ 4151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:47.556234 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:47.556242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:47.556248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:47.556422 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:47.556867 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:47.556871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:47.556873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:47.557181 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:47.557301 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:47.558304 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0809 12:06:47.558487 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:06:47.560299 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.540163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39131 (local address 127.0.0.1:55918)
0809 12:06:47.540985 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.541033 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:47.542768 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:47.543517 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:47.543575 (+ 58us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:47.544142 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:47.544695 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:47.544712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.545744 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.545748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:47.546961 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:47.546973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.547827 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.547835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:47.548055 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:47.548769 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:47.548796 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:47.550957 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:47.555625 (+ 4668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:47.555631 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:47.555643 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:47.556101 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:47.556539 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:47.556542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:47.556544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:47.556732 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:47.557332 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:47.557343 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:47.557843 (+ 500us) client_negotiation.cc:769] Sending connection context
0809 12:06:47.558216 (+ 373us) client_negotiation.cc:240] Negotiation successful
0809 12:06:47.558760 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":365,"threads_started":1}
WARNING: no policy specified for kudu/127.0.147.124@KRBTEST.COM; defaulting to no policy
I20260809 12:06:47.562122 20109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.124@KRBTEST.COM" created.
Aug 09 12:06:47 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
I20260809 12:06:47.577056 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.562795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38286)
0809 12:06:47.563064 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.563083 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:47.563218 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:47.564167 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:47.564171 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:47.564624 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:47.564753 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:47.564760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.566072 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.566078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:47.566905 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:47.566916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.567789 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.567795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:47.568047 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:47.568636 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:47.568652 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:47.571052 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:47.574427 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:47.574435 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:47.574441 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:47.574968 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:47.575406 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:47.575409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:47.575411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:47.575627 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:47.576371 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:47.576375 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:47.576484 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 12:06:47.576594 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 12:06:47.576778 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":22}
I20260809 12:06:47.577564 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.562975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.0.147.125:45421)
0809 12:06:47.563670 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:47.563680 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:47.563714 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:06:47.563770 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:47.563782 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:47.564013 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:47.564172 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:47.564905 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.565937 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.567041 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.567648 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.567780 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:47.571259 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:47.571270 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:47.571282 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:47.571313 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:47.574300 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:47.575118 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:47.575127 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:47.575133 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:47.575286 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:47.575736 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:47.575743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:47.575747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:47.576237 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:47.576452 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:47.576776 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 12:06:47.577054 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":269,"threads_started":1}
I20260809 12:06:47.580121 20109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:06:47.581167 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.580715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49050)
0809 12:06:47.580875 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.581011 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 12:06:47.581403 20044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
I20260809 12:06:47.583033 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.582685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49058)
0809 12:06:47.582868 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.582938 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 12:06:47.583246 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:35457: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab.
Entry for principal kudu/127.0.147.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab.
I20260809 12:06:47.598570 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.124:35457
--webserver_interface=127.0.147.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab
--principal=kudu/127.0.147.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:39131,127.0.147.125:45421,127.0.147.124:35457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:06:47.632537 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } attempt: 1
I20260809 12:06:47.633651 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.633199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49062)
0809 12:06:47.633392 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.633501 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 12:06:47.634644 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.634299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49068)
0809 12:06:47.634472 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.634541 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:06:47.634873 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:35457: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
I20260809 12:06:47.703171 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } attempt: 2
I20260809 12:06:47.704370 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.703907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49078)
0809 12:06:47.704110 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.704210 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 12:06:47.705369 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.705033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49082)
0809 12:06:47.705188 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.705255 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:06:47.705578 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:35457: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
I20260809 12:06:47.718482 19997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } attempt: 5
Aug 09 12:06:47 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
I20260809 12:06:47.732860 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.719217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38298)
0809 12:06:47.719432 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.719454 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:47.719640 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:47.720012 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:47.720016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:47.720370 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:47.720568 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:47.720577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.721931 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.721935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:47.722810 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:47.722822 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.723932 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.723936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:47.724067 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:47.724693 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:47.724710 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:47.726787 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:47.730542 (+ 3755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:47.730546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:47.730549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:47.730921 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:47.731322 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:47.731325 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:47.731328 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:47.731456 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:47.732141 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:47.732145 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:47.732287 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 12:06:47.732402 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 12:06:47.732620 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 12:06:47.733137 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.719389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.0.147.125:45421)
0809 12:06:47.719601 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:47.719607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:47.719632 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:06:47.719761 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:47.719767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:47.719865 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:47.719977 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:47.720715 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.721802 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.722953 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:47.723785 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:47.723952 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:47.726995 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:47.727002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:47.727007 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:47.727046 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:47.730391 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:47.731041 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:47.731046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:47.731050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:47.731211 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:47.731622 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:47.731627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:47.731630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:47.732005 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:47.732169 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:47.732540 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 12:06:47.732765 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:06:47.734299 19997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:06:47.735296 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.734930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49096)
0809 12:06:47.735099 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.735168 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 12:06:47.736290 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.735999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49102)
0809 12:06:47.736128 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.736203 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 12:06:47.736543 19997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:35457: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
I20260809 12:06:47.787777 19997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } attempt: 1
I20260809 12:06:47.789050 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.788569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49104)
0809 12:06:47.788775 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.788886 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 12:06:47.790055 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.789715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49116)
0809 12:06:47.789866 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.789949 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:06:47.790263 19997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:35457: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
I20260809 12:06:47.796803 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } attempt: 3
I20260809 12:06:47.797962 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.797387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49122)
0809 12:06:47.797621 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.797743 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 12:06:47.799258 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.798653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49132)
0809 12:06:47.798941 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.799071 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260809 12:06:47.799571 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:35457: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
I20260809 12:06:47.858497 19997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } attempt: 2
I20260809 12:06:47.859707 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.859148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49148)
0809 12:06:47.859377 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.859464 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 12:06:47.860726 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.860392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49162)
0809 12:06:47.860547 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.860620 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:06:47.860975 19997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:35457: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
W20260809 12:06:47.937033 20120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:47.937309 20120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:47.937510 20120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 12:06:47.942806 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } attempt: 4
I20260809 12:06:47.944053 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.943443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49170)
0809 12:06:47.943730 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.943847 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 12:06:47.945113 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.944744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49182)
0809 12:06:47.944893 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.944977 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:06:47.945362 20120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:06:47.945343 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:35457: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
W20260809 12:06:47.945472 20120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:47.945576 20120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:47.945675 20120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:06:47.945761 20120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:06:47.955265 20120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:39131,127.0.147.125:45421,127.0.147.124:35457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab
--principal=kudu/127.0.147.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.124:35457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.147.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:47.956542 20120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:47.957830 20120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 12:06:47.967202 19997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } attempt: 3
I20260809 12:06:47.968585 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.967906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49198)
0809 12:06:47.968257 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.968362 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260809 12:06:47.969920 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:47.969376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49206)
0809 12:06:47.969688 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:47.969785 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
W20260809 12:06:47.969769 20131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:47.969933 20129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:47.970315 19997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:35457: Network error: Client connection negotiation failed: client connection to 127.0.147.124:35457: connect: Connection refused (error 111)
W20260809 12:06:47.970319 20134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:47.971895 20120 server_base.cc:1034] running on GCE node
I20260809 12:06:47.972735 20120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:47.974210 20120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:47.975443 20120 hybrid_clock.cc:648] HybridClock initialized: now 1786277207975392 us; error 61 us; skew 500 ppm
I20260809 12:06:47.975831 20120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:47 dist-test-slave-sb2z krb5kdc[19887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:47.979211 20120 init.cc:377] Logged in from keytab as kudu/127.0.147.124@KRBTEST.COM (short username kudu)
I20260809 12:06:47.980743 20120 webserver.cc:460] Webserver started at http://127.0.147.124:44489/ using document root <none> and password file <none>
I20260809 12:06:47.981460 20120 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:47.981606 20120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:47.981952 20120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:06:47.985347 20120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/data/instance:
uuid: "d568168c19e44e12a0111b61f5606d56"
format_stamp: "Formatted at 2026-08-09 12:06:47 on dist-test-slave-sb2z"
server_key: "4d1e9dd98a521513b775b5c95f0cf170"
server_key_iv: "ad55ebb0321960e8bfe6bbd985e5902f"
server_key_version: "encryptionkey@0"
I20260809 12:06:47.986200 20120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/wal/instance:
uuid: "d568168c19e44e12a0111b61f5606d56"
format_stamp: "Formatted at 2026-08-09 12:06:47 on dist-test-slave-sb2z"
server_key: "4d1e9dd98a521513b775b5c95f0cf170"
server_key_iv: "ad55ebb0321960e8bfe6bbd985e5902f"
server_key_version: "encryptionkey@0"
I20260809 12:06:47.991482 20120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:06:47.994969 20146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:47.996364 20120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:06:47.996543 20120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/wal
uuid: "d568168c19e44e12a0111b61f5606d56"
format_stamp: "Formatted at 2026-08-09 12:06:47 on dist-test-slave-sb2z"
server_key: "4d1e9dd98a521513b775b5c95f0cf170"
server_key_iv: "ad55ebb0321960e8bfe6bbd985e5902f"
server_key_version: "encryptionkey@0"
I20260809 12:06:47.996762 20120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:48.016063 20120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:48.020606 20120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:48.020972 20120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:48.040187 20120 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.124:35457
I20260809 12:06:48.040246 20227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.124:35457 every 8 connection(s)
I20260809 12:06:48.041793 20120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
I20260809 12:06:48.045573 20228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:48.046790 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 20120
I20260809 12:06:48.046998 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786276990256303-589-0/minicluster-data/master-2/wal/instance
I20260809 12:06:48.047474 589 external_mini_cluster.cc:1428] Setting key 6734b7f3a0783f399d5f9fe37526db5a
I20260809 12:06:48.054993 20228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:06:48 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
Aug 09 12:06:48 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:48.072628 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.049345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.0.147.124:35457)
0809 12:06:48.050243 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.050260 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.050316 (+ 56us) server_negotiation.cc:407] Connection header received
0809 12:06:48.052005 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.052032 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.052410 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.052724 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.054662 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.056080 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.057807 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.058877 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.059079 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.064013 (+ 4934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.064020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.064030 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.064066 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.067917 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.068704 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.068714 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.068723 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.068924 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.069376 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.069382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.069388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.070126 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.070442 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.070842 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.071276 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":617,"threads_started":1}
I20260809 12:06:48.074750 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.057771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55934 (local address 127.0.147.126:39131)
0809 12:06:48.057970 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.057974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.058607 (+ 633us) server_negotiation.cc:407] Connection header received
0809 12:06:48.058820 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.058826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.058900 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.059026 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.060405 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.061592 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.063302 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.064040 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.064143 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.068260 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.068264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.068268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.068300 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.071876 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.072574 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.072583 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.072586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.072772 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.073258 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.073265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.073271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.073604 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.073759 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.074317 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.074488 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:06:48.074966 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.057756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:39131 (local address 127.0.0.1:55934)
0809 12:06:48.058475 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:48.058513 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:48.058682 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:48.059104 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:48.059110 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:48.059950 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:48.060215 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.060223 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.061763 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.061767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.063148 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.063161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.064243 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.064250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.064453 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.065247 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:48.065268 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:48.068048 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:48.072008 (+ 3960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.072017 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.072022 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.072447 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.072902 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.072907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.072911 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.073119 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.073773 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:48.073780 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:48.074010 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 12:06:48.074236 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 12:06:48.074552 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":376,"threads_started":1}
I20260809 12:06:48.076661 20228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:06:48 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
Aug 09 12:06:48 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:48.090268 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.077530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:45421 (local address 127.0.0.1:38304)
0809 12:06:48.077684 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:48.077701 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:48.077827 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:48.078151 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:48.078155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:48.078510 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:48.078642 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.078650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.079635 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.079642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.080499 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.080510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.081402 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.081408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.081624 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.082139 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:48.082155 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:48.084507 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:48.087836 (+ 3329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.087843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.087847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.088223 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.088625 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.088629 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.088635 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.088778 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.089294 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:48.089298 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:48.089480 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 12:06:48.089650 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 12:06:48.089925 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 12:06:48.090273 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.077441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.0.147.125:45421)
0809 12:06:48.077621 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.077627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.077777 (+ 150us) server_negotiation.cc:407] Connection header received
0809 12:06:48.077944 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.077949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.078020 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.078127 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.078795 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.079459 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.080773 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.081231 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.081380 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.084698 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.084703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.084707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.084772 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.087714 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.088354 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.088360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.088363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.088508 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.088894 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.088900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.088903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.089171 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.089336 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.089761 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.089938 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 12:06:48.091801 20228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:06:48.092468 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.076299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55944 (local address 127.0.147.126:39131)
0809 12:06:48.076487 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.076493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.076512 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:06:48.076609 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.076614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.076684 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.076784 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.078285 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.079026 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.080405 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.080816 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.080976 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.086335 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.086344 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.086351 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.086392 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.089379 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.090184 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.090191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.090194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.090354 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.090879 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.090884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.090887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.091235 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.091650 (+ 415us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.091931 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.092170 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 12:06:48 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
Aug 09 12:06:48 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
I20260809 12:06:48.107713 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.092515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49234)
0809 12:06:48.093098 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:48.093117 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:48.093247 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:48.093545 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:48.093549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:48.093972 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:48.094165 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.094178 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.095100 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.095107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.096542 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.096554 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.097522 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.097526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.097708 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.098235 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:48.098256 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:48.100867 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:48.104849 (+ 3982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.104858 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.104864 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.105318 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.105748 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.105752 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.105754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.105890 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.106525 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:48.106530 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:48.106679 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 12:06:48.106840 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 12:06:48.107197 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":1}
I20260809 12:06:48.107743 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.092611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.0.147.124:35457)
0809 12:06:48.092784 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.092790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.093201 (+ 411us) server_negotiation.cc:407] Connection header received
0809 12:06:48.093348 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.093351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.093405 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.093489 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.094296 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.094987 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.096684 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.097366 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.097495 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.101111 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.101120 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.101128 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.101165 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.104691 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.105459 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.105467 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.105473 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.105620 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.106056 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.106065 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.106069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.106375 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.106564 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.106934 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.107358 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":176,"server-negotiator.queue_time_us":62}
I20260809 12:06:48.111950 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.094300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38306 (local address 127.0.147.125:45421)
0809 12:06:48.094480 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.094487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.094555 (+ 68us) server_negotiation.cc:407] Connection header received
0809 12:06:48.094730 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.094735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.094801 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.094900 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.096472 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.097556 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.099380 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.100077 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.100210 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.105851 (+ 5641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.105857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.105862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.105900 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.109005 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.109837 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.109842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.109844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.110013 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.110444 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.110447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.110449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.110717 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.110964 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.111286 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.111485 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:06:48.113854 20228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56: Bootstrap starting.
I20260809 12:06:48.117434 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:06:48.117698 20228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:48.118999 20228 log.cc:824] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:48.122440 20228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56: No bootstrap required, opened a new log
I20260809 12:06:48.131505 20228 raft_consensus.cc:348] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } }
I20260809 12:06:48.132128 20228 raft_consensus.cc:374] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:48.132331 20228 raft_consensus.cc:729] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d568168c19e44e12a0111b61f5606d56, State: Initialized, Role: FOLLOWER
I20260809 12:06:48.133318 20228 consensus_queue.cc:260] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } }
I20260809 12:06:48.145704 20228 sys_catalog.cc:564] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:06:48.146126 20245 sys_catalog.cc:455] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.147547 19997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } attempt: 4
I20260809 12:06:48.149753 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.123455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.0.147.125:45421)
0809 12:06:48.123685 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.123691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.128512 (+ 4821us) server_negotiation.cc:407] Connection header received
0809 12:06:48.128764 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.128770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.128851 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.128929 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.131780 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.133216 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.136110 (+ 2894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.137018 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.137140 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.141690 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.141697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.141702 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.141743 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.145252 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.146231 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.146237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.146240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.146463 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.147012 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.147019 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.147024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.147466 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.147792 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.149084 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.149337 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260809 12:06:48.150331 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.123861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.0.147.126:39131)
0809 12:06:48.124038 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.124045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.128215 (+ 4170us) server_negotiation.cc:407] Connection header received
0809 12:06:48.128514 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.128520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.128601 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.128670 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.130963 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.132068 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.134336 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.135248 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.135383 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.140797 (+ 5414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.140807 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.140815 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.140872 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.145465 (+ 4593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.147315 (+ 1850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.147323 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.147329 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.147566 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.148127 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.148133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.148136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.148530 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.148735 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.149522 (+ 787us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.149835 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:06:48.151510 20245 sys_catalog.cc:458] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:06:48.158206 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.128596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.0.147.124:35457)
0809 12:06:48.128807 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.128813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.139029 (+ 10216us) server_negotiation.cc:407] Connection header received
0809 12:06:48.139561 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.139567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.139655 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.139738 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.141922 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.143088 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.145067 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.146052 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.146210 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.150072 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.150080 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.150085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.150133 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.154335 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.155231 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.155239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.155246 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.155450 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.156106 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.156111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.156115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.156500 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.156758 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.157352 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.157645 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 12:06:48 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
W20260809 12:06:48.168394 20262 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:06:48.168591 20262 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:06:48.170250 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.148136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49248)
0809 12:06:48.148491 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:48.148510 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:48.148605 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:48.153303 (+ 4698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:48.153308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:48.153663 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:48.153815 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.153823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.155516 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.155553 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.157016 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.157033 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.158511 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.158518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.158664 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.159241 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:48.159268 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:48.162134 (+ 2866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:48.167176 (+ 5042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.167183 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.167187 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.167752 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.168324 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.168332 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.168337 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.168524 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.169186 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:48.169192 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:48.169404 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 12:06:48.169564 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 12:06:48.169848 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238}
I20260809 12:06:48.171615 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.148792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49248 (local address 127.0.147.124:35457)
0809 12:06:48.152892 (+ 4100us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.152896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.152919 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:06:48.152997 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.153005 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.153080 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.153935 (+ 855us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.154195 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.155379 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.157171 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.158140 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.158290 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.162329 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.162335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.162349 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.162390 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.166961 (+ 4571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.167967 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.167971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.167974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.168139 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.168676 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.168680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.168683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.169043 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.169221 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.170910 (+ 1689us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.171150 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3985,"thread_start_us":3787,"threads_started":1}
I20260809 12:06:48.164810 20120 master_runner.cc:186] Error getting master registration for 127.0.147.126:39131: OK, instance_id { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" instance_seqno: 1786277207005186 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:06:48.176393 20120 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:06:48.176961 19997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a: Bootstrap starting.
I20260809 12:06:48.181056 19997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:48.182457 19997 log.cc:824] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:48.185896 19997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a: No bootstrap required, opened a new log
I20260809 12:06:48.197237 19997 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } }
I20260809 12:06:48.197654 19997 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:48.197781 19997 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2a9fb0550704c9da616b59f629eb44a, State: Initialized, Role: FOLLOWER
I20260809 12:06:48.198464 19997 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } }
I20260809 12:06:48.199407 20270 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.199816 20270 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:06:48.200881 19997 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [sys.catalog]: configured and running, proceeding with master startup.
W20260809 12:06:48.212054 20290 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:06:48.212286 20290 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:06:48.211175 19903 master_runner.cc:186] Error getting master registration for 127.0.147.125:45421: OK, instance_id { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" instance_seqno: 1786277207519478 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:06:48.222886 19903 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 12:06:48.244601 19932 tablet.cc:2369] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:06:48.244596 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } attempt: 5
Aug 09 12:06:48 dist-test-slave-sb2z krb5kdc[19887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
I20260809 12:06:48.262508 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.245237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:35457 (local address 127.0.0.1:49262)
0809 12:06:48.245558 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:48.245579 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:48.245707 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:48.246127 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:48.246134 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:48.246697 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:48.246942 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.246955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.248456 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.248463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.249925 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:48.249942 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.250950 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.250956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:48.251174 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.251897 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:48.251930 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:48.254778 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:48.259519 (+ 4741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.259555 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.259560 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.260120 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.260590 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:48.260596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:48.260599 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:48.260772 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:48.261459 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:48.261464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:48.261631 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 12:06:48.261806 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 12:06:48.262110 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260809 12:06:48.262871 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:48.245589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.0.147.124:35457)
0809 12:06:48.245779 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:48.245785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:48.245806 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:06:48.245872 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:48.245877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:48.245971 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:48.246082 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:48.247120 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.248291 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.250107 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:48.250785 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:48.250916 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:48.255034 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:48.255041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:48.255046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:48.255092 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:48.259346 (+ 4254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.260273 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.260279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.260284 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.260450 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:48.260893 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:48.260910 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:48.260913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:48.261324 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:48.261474 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:48.261933 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 12:06:48.262181 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:06:48.268800 20109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40: Bootstrap starting.
I20260809 12:06:48.273016 20109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40: Neither blocks nor log segments found. Creating new log.
W20260809 12:06:48.273010 20156 tablet.cc:2369] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:06:48.274350 20109 log.cc:824] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:48.278460 20109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40: No bootstrap required, opened a new log
I20260809 12:06:48.289525 20109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } }
I20260809 12:06:48.289930 20109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:48.290073 20109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9736a4f2c3d049b1b7aaef6cbc065e40, State: Initialized, Role: FOLLOWER
I20260809 12:06:48.290707 20109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } }
I20260809 12:06:48.293064 20109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:06:48.291710 20294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.295929 20294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:06:48.309466 20009 master_runner.cc:263] No leader master found from master 9736a4f2c3d049b1b7aaef6cbc065e40
I20260809 12:06:48.309605 20009 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 12:06:48.315835 20326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:06:48.316071 20326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:06:48.341528 20270 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:06:48.341843 20270 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } }
I20260809 12:06:48.344494 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2a9fb0550704c9da616b59f629eb44a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d568168c19e44e12a0111b61f5606d56" is_pre_election: true
I20260809 12:06:48.345155 20191 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2a9fb0550704c9da616b59f629eb44a in term 0.
I20260809 12:06:48.346468 20074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2a9fb0550704c9da616b59f629eb44a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" is_pre_election: true
I20260809 12:06:48.347092 20074 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2a9fb0550704c9da616b59f629eb44a in term 0.
I20260809 12:06:48.347859 19926 leader_election.cc:302] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2a9fb0550704c9da616b59f629eb44a, d568168c19e44e12a0111b61f5606d56; no voters:
I20260809 12:06:48.343462 20270 leader_election.cc:288] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9736a4f2c3d049b1b7aaef6cbc065e40 (127.0.147.125:45421), d568168c19e44e12a0111b61f5606d56 (127.0.147.124:35457)
I20260809 12:06:48.352413 20270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:06:48.352560 20270 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:06:48.352658 20270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:48.357615 20270 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } }
I20260809 12:06:48.359499 20074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2a9fb0550704c9da616b59f629eb44a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40"
I20260809 12:06:48.359741 20074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:48.364712 20074 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2a9fb0550704c9da616b59f629eb44a in term 1.
I20260809 12:06:48.365199 19930 leader_election.cc:302] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9736a4f2c3d049b1b7aaef6cbc065e40, c2a9fb0550704c9da616b59f629eb44a; no voters:
I20260809 12:06:48.366082 20270 leader_election.cc:288] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [CANDIDATE]: Term 1 election: Requested vote from peers 9736a4f2c3d049b1b7aaef6cbc065e40 (127.0.147.125:45421), d568168c19e44e12a0111b61f5606d56 (127.0.147.124:35457)
I20260809 12:06:48.366582 20270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:06:48.366973 20270 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [term 1 LEADER]: Becoming Leader. State: Replica: c2a9fb0550704c9da616b59f629eb44a, State: Running, Role: LEADER
I20260809 12:06:48.368054 20270 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } }
I20260809 12:06:48.382321 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2a9fb0550704c9da616b59f629eb44a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d568168c19e44e12a0111b61f5606d56"
I20260809 12:06:48.382587 20191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:48.386219 20191 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2a9fb0550704c9da616b59f629eb44a in term 1.
I20260809 12:06:48.389796 20328 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2a9fb0550704c9da616b59f629eb44a. Latest consensus state: current_term: 1 leader_uuid: "c2a9fb0550704c9da616b59f629eb44a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.390334 20328 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:48.397539 20330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:06:48.400256 20330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:06:48.421614 20191 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [term 1 FOLLOWER]: Refusing update from remote peer c2a9fb0550704c9da616b59f629eb44a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:06:48.422712 20328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:06:48.430096 20074 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [term 1 FOLLOWER]: Refusing update from remote peer c2a9fb0550704c9da616b59f629eb44a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:06:48.431039 20328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:06:48.456998 20294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2a9fb0550704c9da616b59f629eb44a. Latest consensus state: current_term: 1 leader_uuid: "c2a9fb0550704c9da616b59f629eb44a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.457331 20294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:06:48.475440 20331 mvcc.cc:205] Tried to move back new op lower bound from 7316591445683232768 to 7316591445486587904. Current Snapshot: MvccSnapshot[applied={T|T < 7316591445683232768}]
I20260809 12:06:48.491071 20330 catalog_manager.cc:1347] Generated new cluster ID: 8373aa2cfb2142e08d147d86d112782b
I20260809 12:06:48.491196 20330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:06:48.509017 20294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2a9fb0550704c9da616b59f629eb44a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.509320 20294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:06:48.511298 20270 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2a9fb0550704c9da616b59f629eb44a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.511650 20270 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:48.529175 20245 sys_catalog.cc:455] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2a9fb0550704c9da616b59f629eb44a. Latest consensus state: current_term: 1 leader_uuid: "c2a9fb0550704c9da616b59f629eb44a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.529543 20245 sys_catalog.cc:458] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:06:48.536864 20328 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2a9fb0550704c9da616b59f629eb44a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.537173 20328 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:48.539217 20332 mvcc.cc:205] Tried to move back new op lower bound from 7316591445683232768 to 7316591445486587904. Current Snapshot: MvccSnapshot[applied={T|T < 7316591445683232768}]
I20260809 12:06:48.547940 20245 sys_catalog.cc:455] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2a9fb0550704c9da616b59f629eb44a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2a9fb0550704c9da616b59f629eb44a" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39131 } } peers { permanent_uuid: "9736a4f2c3d049b1b7aaef6cbc065e40" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 45421 } } peers { permanent_uuid: "d568168c19e44e12a0111b61f5606d56" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 35457 } } }
I20260809 12:06:48.548247 20245 sys_catalog.cc:458] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:06:48.581923 20330 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:06:48.584482 20330 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:06:48.612066 20330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2a9fb0550704c9da616b59f629eb44a: Generated new TSK 0
I20260809 12:06:48.613407 20330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:06:49.170529 20262 catalog_manager.cc:1259] Loaded cluster ID: 8373aa2cfb2142e08d147d86d112782b
I20260809 12:06:49.170626 20262 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56: loading cluster ID for follower catalog manager: success
I20260809 12:06:49.172971 20262 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56: acquiring CA information for follower catalog manager: success
I20260809 12:06:49.174839 20262 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d568168c19e44e12a0111b61f5606d56: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:06:49.318926 20326 catalog_manager.cc:1259] Loaded cluster ID: 8373aa2cfb2142e08d147d86d112782b
I20260809 12:06:49.319075 20326 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40: loading cluster ID for follower catalog manager: success
I20260809 12:06:49.323309 20326 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40: acquiring CA information for follower catalog manager: success
I20260809 12:06:49.325659 20326 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9736a4f2c3d049b1b7aaef6cbc065e40: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:06:49.782613 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 19903
I20260809 12:06:49.784228 19992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:49.897214 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 19903
I20260809 12:06:49.930724 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 20009
I20260809 12:06:49.931977 20102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:50.042223 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 20009
I20260809 12:06:50.069859 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 20120
I20260809 12:06:50.071029 20223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:50.182057 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 20120
2026-08-09T12:06:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5250 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:06:50 dist-test-slave-sb2z krb5kdc[20362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:06:50 dist-test-slave-sb2z krb5kdc[20362](info): set up 2 sockets
Aug 09 12:06:50 dist-test-slave-sb2z krb5kdc[20362](info): commencing operation
krb5kdc: starting...
W20260809 12:06:51.507616 589 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:06:51 dist-test-slave-sb2z krb5kdc[20362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:06:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:06:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:06:51.810978 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45117
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:45117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:06:52.153563 20379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:52.153893 20379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:52.154069 20379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:52.161118 20379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:06:52.161269 20379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:52.161394 20379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:52.161491 20379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:06:52.161582 20379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:06:52.171213 20379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:45117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:52.172531 20379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:52.173856 20379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:52.185847 20386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:52.185891 20387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:52.185926 20389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:52.186838 20379 server_base.cc:1034] running on GCE node
I20260809 12:06:52.187678 20379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:52.189081 20379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:52.190335 20379 hybrid_clock.cc:648] HybridClock initialized: now 1786277212190294 us; error 45 us; skew 500 ppm
I20260809 12:06:52.190689 20379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:52 dist-test-slave-sb2z krb5kdc[20362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277212, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:52.199329 20379 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:06:52.200991 20379 webserver.cc:460] Webserver started at http://127.0.147.126:37009/ using document root <none> and password file <none>
I20260809 12:06:52.201699 20379 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:52.201848 20379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:52.202178 20379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:06:52.205554 20379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "159b17b192cd46d7a9a106be69c0e4ec"
format_stamp: "Formatted at 2026-08-09 12:06:52 on dist-test-slave-sb2z"
server_key: "04b86a7f8568f40c4a4ae3c819f92391"
server_key_iv: "da2cd1e037eae0f4df99d8083bcf6155"
server_key_version: "encryptionkey@0"
I20260809 12:06:52.206393 20379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "159b17b192cd46d7a9a106be69c0e4ec"
format_stamp: "Formatted at 2026-08-09 12:06:52 on dist-test-slave-sb2z"
server_key: "04b86a7f8568f40c4a4ae3c819f92391"
server_key_iv: "da2cd1e037eae0f4df99d8083bcf6155"
server_key_version: "encryptionkey@0"
I20260809 12:06:52.211789 20379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 12:06:52.215293 20402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:52.216637 20379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:06:52.216858 20379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "159b17b192cd46d7a9a106be69c0e4ec"
format_stamp: "Formatted at 2026-08-09 12:06:52 on dist-test-slave-sb2z"
server_key: "04b86a7f8568f40c4a4ae3c819f92391"
server_key_iv: "da2cd1e037eae0f4df99d8083bcf6155"
server_key_version: "encryptionkey@0"
I20260809 12:06:52.217070 20379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:52.256404 20379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:52.260462 20379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:52.260758 20379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:52.280153 20379 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:45117
I20260809 12:06:52.280146 20478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:45117 every 8 connection(s)
I20260809 12:06:52.281538 20379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:06:52.285171 20479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:52.290504 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 20379
I20260809 12:06:52.290709 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:06:52.291349 589 external_mini_cluster.cc:1428] Setting key 2e924055af42de266060c9e233d309bb
I20260809 12:06:52.295877 20479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec: Bootstrap starting.
I20260809 12:06:52.298949 20479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:52.300209 20479 log.cc:824] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:52.304327 20479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec: No bootstrap required, opened a new log
Aug 09 12:06:52 dist-test-slave-sb2z krb5kdc[20362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:52.314864 20479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159b17b192cd46d7a9a106be69c0e4ec" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45117 } }
I20260809 12:06:52.315310 20479 raft_consensus.cc:374] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:52.315424 20479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 159b17b192cd46d7a9a106be69c0e4ec, State: Initialized, Role: FOLLOWER
I20260809 12:06:52.316159 20479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159b17b192cd46d7a9a106be69c0e4ec" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45117 } }
I20260809 12:06:52.316447 20479 raft_consensus.cc:388] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:06:52.316556 20483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:52.293301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.0.147.126:45117)
0809 12:06:52.294101 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:52.294112 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:52.294150 (+ 38us) server_negotiation.cc:407] Connection header received
0809 12:06:52.295897 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:52.295918 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:52.296303 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:52.296680 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:52.298194 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:52.299640 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:52.301590 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:52.302501 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:52.302629 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:52.307821 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:52.307836 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:52.307856 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:52.307906 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:52.312038 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:52.312752 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:52.312765 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:52.312775 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:52.313009 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:52.313371 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:52.313376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:52.313380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:52.314095 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:52.314367 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:52.314751 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 12:06:52.315176 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":349,"threads_started":1}
I20260809 12:06:52.316628 20479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:06:52.316922 20479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:52.319370 20479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159b17b192cd46d7a9a106be69c0e4ec" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45117 } }
I20260809 12:06:52.319954 20479 leader_election.cc:302] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 159b17b192cd46d7a9a106be69c0e4ec; no voters:
I20260809 12:06:52.320657 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:06:52.320762 20479 leader_election.cc:288] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:06:52.320871 20487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:06:52.322511 20487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [term 1 LEADER]: Becoming Leader. State: Replica: 159b17b192cd46d7a9a106be69c0e4ec, State: Running, Role: LEADER
I20260809 12:06:52.322962 20479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:06:52.323040 20487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159b17b192cd46d7a9a106be69c0e4ec" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45117 } }
I20260809 12:06:52.327211 20488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "159b17b192cd46d7a9a106be69c0e4ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159b17b192cd46d7a9a106be69c0e4ec" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45117 } } }
I20260809 12:06:52.327682 20488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:52.327191 20489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 159b17b192cd46d7a9a106be69c0e4ec. Latest consensus state: current_term: 1 leader_uuid: "159b17b192cd46d7a9a106be69c0e4ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159b17b192cd46d7a9a106be69c0e4ec" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45117 } } }
I20260809 12:06:52.328192 20489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:52.337077 20497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:06:52.342904 20497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:06:52.350939 20497 catalog_manager.cc:1347] Generated new cluster ID: bcf63d804477475dba6813c1b2dccf13
I20260809 12:06:52.351058 20497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:06:52.377643 20497 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:06:52.378935 20497 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:06:52.385196 20497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 159b17b192cd46d7a9a106be69c0e4ec: Generated new TSK 0
I20260809 12:06:52.385946 20497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:06:52.434938 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 20379
I20260809 12:06:52.436307 20473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:52.534850 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 20379
2026-08-09T12:06:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2347 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:06:52 dist-test-slave-sb2z krb5kdc[20520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:06:52 dist-test-slave-sb2z krb5kdc[20520](info): set up 2 sockets
Aug 09 12:06:52 dist-test-slave-sb2z krb5kdc[20520](info): commencing operation
krb5kdc: starting...
W20260809 12:06:53.844713 589 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.006s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:06:54 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:06:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:06:54.146487 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38449
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:38449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:06:54.486763 20541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:54.487104 20541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:54.487272 20541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:54.494442 20541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:06:54.494587 20541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:54.494694 20541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:54.494799 20541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:06:54.494885 20541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:06:54.504916 20541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:38449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:54.506311 20541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:54.507725 20541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:54.519713 20556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:54.519800 20549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:54.519758 20550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:54.521019 20541 server_base.cc:1034] running on GCE node
I20260809 12:06:54.521765 20541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:54.523099 20541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:54.524298 20541 hybrid_clock.cc:648] HybridClock initialized: now 1786277214524259 us; error 37 us; skew 500 ppm
I20260809 12:06:54.524639 20541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:54 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:54.527856 20541 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:06:54.529361 20541 webserver.cc:460] Webserver started at http://127.0.147.126:33733/ using document root <none> and password file <none>
I20260809 12:06:54.530009 20541 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:54.530153 20541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:54.530447 20541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:06:54.533385 20541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "a94b6cb3467e46a59938bd85d4e0a8d1"
format_stamp: "Formatted at 2026-08-09 12:06:54 on dist-test-slave-sb2z"
server_key: "5e2ac90aaab6ce0a9a9ac6e1f119f7e6"
server_key_iv: "dec5db35effd3818ea46cda0a5946376"
server_key_version: "encryptionkey@0"
I20260809 12:06:54.534137 20541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "a94b6cb3467e46a59938bd85d4e0a8d1"
format_stamp: "Formatted at 2026-08-09 12:06:54 on dist-test-slave-sb2z"
server_key: "5e2ac90aaab6ce0a9a9ac6e1f119f7e6"
server_key_iv: "dec5db35effd3818ea46cda0a5946376"
server_key_version: "encryptionkey@0"
I20260809 12:06:54.539043 20541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:06:54.542490 20565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:54.543856 20541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:06:54.544070 20541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "a94b6cb3467e46a59938bd85d4e0a8d1"
format_stamp: "Formatted at 2026-08-09 12:06:54 on dist-test-slave-sb2z"
server_key: "5e2ac90aaab6ce0a9a9ac6e1f119f7e6"
server_key_iv: "dec5db35effd3818ea46cda0a5946376"
server_key_version: "encryptionkey@0"
I20260809 12:06:54.544265 20541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:54.569437 20541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:54.573953 20541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:54.574259 20541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:54.593618 20541 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:38449
I20260809 12:06:54.593650 20645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:38449 every 8 connection(s)
I20260809 12:06:54.594985 20541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:06:54.598666 20648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:54.605365 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 20541
I20260809 12:06:54.605616 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:06:54.606272 589 external_mini_cluster.cc:1428] Setting key 7400e320809ce420b0b0eccbdb33ddcc
I20260809 12:06:54.609660 20648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Bootstrap starting.
I20260809 12:06:54.613662 20648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:54.614984 20648 log.cc:824] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:54.617794 20648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: No bootstrap required, opened a new log
Aug 09 12:06:54 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:54.625576 20648 raft_consensus.cc:348] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:06:54.625959 20648 raft_consensus.cc:374] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:54.626075 20648 raft_consensus.cc:729] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94b6cb3467e46a59938bd85d4e0a8d1, State: Initialized, Role: FOLLOWER
I20260809 12:06:54.626789 20648 consensus_queue.cc:260] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:06:54.627127 20648 raft_consensus.cc:388] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:06:54.627274 20648 raft_consensus.cc:482] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:06:54.627497 20648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:54.630124 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:54.608326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54638 (local address 127.0.147.126:38449)
0809 12:06:54.609129 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:54.609147 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:54.609209 (+ 62us) server_negotiation.cc:407] Connection header received
0809 12:06:54.611758 (+ 2549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:54.611789 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:54.612281 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:54.612713 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:54.614135 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:54.616046 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:54.617400 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:54.618167 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:54.618377 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:54.622585 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:54.622592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:54.622610 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:54.622644 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:54.625852 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:54.626721 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:54.626727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:54.626732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:54.626908 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:54.627349 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:54.627352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:54.627354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:54.628022 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:54.628274 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:54.628763 (+ 489us) server_negotiation.cc:299] Negotiation successful
0809 12:06:54.629147 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":322,"threads_started":1}
I20260809 12:06:54.630369 20648 raft_consensus.cc:504] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:06:54.630923 20648 leader_election.cc:302] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a94b6cb3467e46a59938bd85d4e0a8d1; no voters:
I20260809 12:06:54.631677 20648 leader_election.cc:288] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:06:54.631789 20653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:06:54.633483 20653 raft_consensus.cc:686] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 1 LEADER]: Becoming Leader. State: Replica: a94b6cb3467e46a59938bd85d4e0a8d1, State: Running, Role: LEADER
I20260809 12:06:54.634011 20648 sys_catalog.cc:564] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:06:54.634106 20653 consensus_queue.cc:237] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:06:54.636802 20655 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94b6cb3467e46a59938bd85d4e0a8d1. Latest consensus state: current_term: 1 leader_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } } }
I20260809 12:06:54.637157 20655 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:54.637693 20654 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } } }
I20260809 12:06:54.638059 20654 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:54.647272 20664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:06:54.652916 20664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:06:54.662786 20664 catalog_manager.cc:1347] Generated new cluster ID: 8aa1c7b4b56d477d886a3f456e933348
I20260809 12:06:54.662914 20664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:06:54.682366 20664 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:06:54.683872 20664 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:06:54.691917 20664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Generated new TSK 0
I20260809 12:06:54.692736 20664 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:06:54.775054 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38449
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:06:55.122108 20687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:55.122372 20687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:55.122469 20687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:55.129726 20687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:55.129838 20687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:55.130024 20687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:06:55.140305 20687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:55.142047 20687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:55.143380 20687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:55.156924 20696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:55.157028 20700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:55.157949 20687 server_base.cc:1034] running on GCE node
W20260809 12:06:55.158124 20697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:55.158830 20687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:55.159734 20687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:55.160946 20687 hybrid_clock.cc:648] HybridClock initialized: now 1786277215160904 us; error 32 us; skew 500 ppm
I20260809 12:06:55.161262 20687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:55 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277215, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:55.164458 20687 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:06:55.165812 20687 webserver.cc:460] Webserver started at http://127.0.147.65:44595/ using document root <none> and password file <none>
I20260809 12:06:55.166484 20687 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:55.166635 20687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:55.166958 20687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:06:55.169889 20687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "4f032bebe61649a79978794ee4f1c1c2"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "68852ec8a895eb21bacb0c8a2fad971d"
server_key_iv: "36e27e5a9d2e4f497b8799d9f6e3de44"
server_key_version: "encryptionkey@0"
I20260809 12:06:55.170703 20687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "4f032bebe61649a79978794ee4f1c1c2"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "68852ec8a895eb21bacb0c8a2fad971d"
server_key_iv: "36e27e5a9d2e4f497b8799d9f6e3de44"
server_key_version: "encryptionkey@0"
I20260809 12:06:55.175820 20687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:06:55.179175 20711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:55.180435 20687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:06:55.180615 20687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "4f032bebe61649a79978794ee4f1c1c2"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "68852ec8a895eb21bacb0c8a2fad971d"
server_key_iv: "36e27e5a9d2e4f497b8799d9f6e3de44"
server_key_version: "encryptionkey@0"
I20260809 12:06:55.180814 20687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:55.220928 20687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:55.225178 20687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:55.225503 20687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:55.226550 20687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:55.227959 20687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:55.228076 20687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:55.228240 20687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:55.228320 20687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:55.262681 20687 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:36953
I20260809 12:06:55.262769 20881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:36953 every 8 connection(s)
I20260809 12:06:55.263978 20687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:06:55.265319 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 20687
I20260809 12:06:55.265542 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:06:55.266443 589 external_mini_cluster.cc:1428] Setting key 42af04e282bfc10b90e126a00587bd37
Aug 09 12:06:55 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277215, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:55.284759 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:55.267240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:44805 (local address 127.0.147.126:38449)
0809 12:06:55.267892 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:55.267896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:55.268515 (+ 619us) server_negotiation.cc:407] Connection header received
0809 12:06:55.270476 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:55.270481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:55.270576 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:55.270696 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:55.272536 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:55.273357 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:55.274926 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:55.275333 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:55.275478 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:55.278951 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:55.278959 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:55.278968 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:55.278999 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:55.281702 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:55.282415 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:55.282421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:55.282426 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:55.282549 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:55.282982 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:55.282986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:55.282989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:55.283251 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:55.283422 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:55.284171 (+ 749us) server_negotiation.cc:299] Negotiation successful
0809 12:06:55.284392 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":405,"threads_started":1}
I20260809 12:06:55.285737 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:55.267352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:44805)
0809 12:06:55.268334 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:55.268407 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:55.270216 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:55.270923 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:55.270931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:55.271434 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:55.272221 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:55.272248 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:55.273515 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:55.273520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:55.274760 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:55.274773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:55.275612 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:55.275620 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:55.275841 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:55.276553 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:55.276581 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:55.278740 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:55.281843 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:55.281852 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:55.281866 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:55.282286 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:55.282662 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:55.282668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:55.282671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:55.282860 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:55.283396 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:55.283403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:55.283786 (+ 383us) client_negotiation.cc:769] Sending connection context
0809 12:06:55.284041 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 12:06:55.284550 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":552,"threads_started":1}
I20260809 12:06:55.287298 20882 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38449
I20260809 12:06:55.287642 20882 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:55.288341 20882 heartbeater.cc:511] Master 127.0.147.126:38449 requested a full tablet report, sending...
I20260809 12:06:55.290583 20592 ts_manager.cc:194] Registered new tserver with Master: 4f032bebe61649a79978794ee4f1c1c2 (127.0.147.65:36953)
I20260809 12:06:55.292804 20592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:44805
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:06:55.338673 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38449
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:06:55.689092 20889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:55.689373 20889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:55.689483 20889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:55.696496 20889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:55.696609 20889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:55.696718 20889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:06:55.706164 20889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:55.707355 20889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:55.708671 20889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:55.721966 20896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:55.722082 20895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:55.722043 20898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:55.723083 20889 server_base.cc:1034] running on GCE node
I20260809 12:06:55.723803 20889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:55.724740 20889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:55.726002 20889 hybrid_clock.cc:648] HybridClock initialized: now 1786277215725963 us; error 46 us; skew 500 ppm
I20260809 12:06:55.726321 20889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:55 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277215, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:55.729620 20889 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:06:55.731197 20889 webserver.cc:460] Webserver started at http://127.0.147.66:46157/ using document root <none> and password file <none>
I20260809 12:06:55.731948 20889 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:55.732102 20889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:55.732398 20889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:06:55.735337 20889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "bcf3812d69105a1bcb0d8606ba6a1de3"
server_key_iv: "3fdd12260240e7a5fe47d766eb79b997"
server_key_version: "encryptionkey@0"
I20260809 12:06:55.736188 20889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "bcf3812d69105a1bcb0d8606ba6a1de3"
server_key_iv: "3fdd12260240e7a5fe47d766eb79b997"
server_key_version: "encryptionkey@0"
I20260809 12:06:55.741189 20889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:06:55.744900 20908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:55.746178 20889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 12:06:55.746366 20889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "bcf3812d69105a1bcb0d8606ba6a1de3"
server_key_iv: "3fdd12260240e7a5fe47d766eb79b997"
server_key_version: "encryptionkey@0"
I20260809 12:06:55.746589 20889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:55.765976 20889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:55.770058 20889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:55.770361 20889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:55.771365 20889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:55.772773 20889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:55.772878 20889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:55.773041 20889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:55.773118 20889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:55.807399 20889 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:36701
I20260809 12:06:55.807447 21069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:36701 every 8 connection(s)
I20260809 12:06:55.808748 20889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:06:55.809195 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 20889
I20260809 12:06:55.809409 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:06:55.810086 589 external_mini_cluster.cc:1428] Setting key 96d9ab07433a7031e127ac2c904037c9
Aug 09 12:06:55 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277215, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:55.833729 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:55.811396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:44199 (local address 127.0.147.126:38449)
0809 12:06:55.811926 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:55.811932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:55.813077 (+ 1145us) server_negotiation.cc:407] Connection header received
0809 12:06:55.815935 (+ 2858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:55.815939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:55.816005 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:55.816099 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:55.818190 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:55.818941 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:55.821475 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:55.821902 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:55.821999 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:55.826207 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:55.826212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:55.826215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:55.826245 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:55.830316 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:55.831063 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:55.831070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:55.831076 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:55.831241 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:55.831746 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:55.831754 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:55.831760 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:55.832136 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:55.832306 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:55.832951 (+ 645us) server_negotiation.cc:299] Negotiation successful
0809 12:06:55.833202 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":275,"threads_started":1}
I20260809 12:06:55.834324 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:55.812055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:44199)
0809 12:06:55.812893 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:55.812945 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:55.815662 (+ 2717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:55.816367 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:55.816381 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:55.817175 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:55.817999 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:55.818030 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:55.819111 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:55.819120 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:55.821333 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:55.821350 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:55.822140 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:55.822150 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:55.822467 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:55.823236 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:55.823269 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:55.825955 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:55.830475 (+ 4520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:55.830482 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:55.830495 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:55.830918 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:55.831358 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:55.831361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:55.831363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:55.831624 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:55.832268 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:55.832273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:55.832584 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 12:06:55.832812 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 12:06:55.833200 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":328,"threads_started":1}
I20260809 12:06:55.835918 21070 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38449
I20260809 12:06:55.836269 21070 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:55.837162 21070 heartbeater.cc:511] Master 127.0.147.126:38449 requested a full tablet report, sending...
I20260809 12:06:55.838889 20592 ts_manager.cc:194] Registered new tserver with Master: 3a3cf7ebe58f4c50ae303d5a5e404e10 (127.0.147.66:36701)
I20260809 12:06:55.840232 20592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:44199
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:06:55.882735 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38449
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:06:56.224097 21078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:56.224401 21078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:56.224582 21078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:56.231626 21078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:56.231765 21078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:56.231949 21078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:06:56.241659 21078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:56.242892 21078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:56.244217 21078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:56.257917 21083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:56.257830 21087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:56.257858 21084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:56.258951 21078 server_base.cc:1034] running on GCE node
I20260809 12:06:56.259647 21078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:56.260751 21078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:56.261979 21078 hybrid_clock.cc:648] HybridClock initialized: now 1786277216261924 us; error 40 us; skew 500 ppm
I20260809 12:06:56.262357 21078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:56 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:56.266423 21078 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:06:56.268066 21078 webserver.cc:460] Webserver started at http://127.0.147.67:37755/ using document root <none> and password file <none>
I20260809 12:06:56.268736 21078 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:56.268894 21078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:56.269191 21078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:06:56.272073 21078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "0986c1d29f814cae913e5491624c982d"
format_stamp: "Formatted at 2026-08-09 12:06:56 on dist-test-slave-sb2z"
server_key: "3d13094249bf09c787d993b60db1c5eb"
server_key_iv: "437be60854ece156a43947fb11417430"
server_key_version: "encryptionkey@0"
I20260809 12:06:56.272868 21078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "0986c1d29f814cae913e5491624c982d"
format_stamp: "Formatted at 2026-08-09 12:06:56 on dist-test-slave-sb2z"
server_key: "3d13094249bf09c787d993b60db1c5eb"
server_key_iv: "437be60854ece156a43947fb11417430"
server_key_version: "encryptionkey@0"
I20260809 12:06:56.277925 21078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:06:56.281466 21097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:56.282830 21078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:06:56.283012 21078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "0986c1d29f814cae913e5491624c982d"
format_stamp: "Formatted at 2026-08-09 12:06:56 on dist-test-slave-sb2z"
server_key: "3d13094249bf09c787d993b60db1c5eb"
server_key_iv: "437be60854ece156a43947fb11417430"
server_key_version: "encryptionkey@0"
I20260809 12:06:56.283215 21078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:56.295663 20882 heartbeater.cc:503] Master 127.0.147.126:38449 was elected leader, sending a full tablet report...
I20260809 12:06:56.299461 21078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:56.303902 21078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:56.304186 21078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:56.305160 21078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:56.306468 21078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:56.306581 21078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:56.306737 21078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:56.306813 21078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:56.349485 21078 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:40953
I20260809 12:06:56.349534 21274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:40953 every 8 connection(s)
I20260809 12:06:56.350886 21078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:06:56.352418 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 21078
I20260809 12:06:56.352641 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:06:56.353144 589 external_mini_cluster.cc:1428] Setting key 17392368639523edadf3b99c279befc1
Aug 09 12:06:56 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:56.370986 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.353712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:57285 (local address 127.0.147.126:38449)
0809 12:06:56.354226 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.354232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.355516 (+ 1284us) server_negotiation.cc:407] Connection header received
0809 12:06:56.357391 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.357395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.357495 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.357591 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:56.359046 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.359921 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.361444 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.361829 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.361925 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.365267 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:56.365271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:56.365275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:56.365307 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:56.368262 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:56.369026 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:56.369030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:56.369032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:56.369160 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:56.369555 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:56.369558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:56.369571 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:56.369822 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:56.369937 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.370542 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.370717 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":273,"threads_started":1}
I20260809 12:06:56.371963 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.354366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:57285)
0809 12:06:56.355353 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:56.355407 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:56.357132 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:56.357794 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:56.357801 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:56.358330 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:56.358864 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.358882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.360078 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.360082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.361310 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.361322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.362009 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.362014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.362219 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.362814 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:56.362834 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:56.365071 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:56.368390 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:56.368401 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:56.368416 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:56.368921 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:56.369253 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:56.369256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:56.369258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:56.369453 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:56.369938 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:56.369944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:56.370236 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 12:06:56.370454 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 12:06:56.370824 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":460,"threads_started":1}
I20260809 12:06:56.373243 21275 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38449
I20260809 12:06:56.373500 21275 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:56.374185 21275 heartbeater.cc:511] Master 127.0.147.126:38449 requested a full tablet report, sending...
I20260809 12:06:56.375506 20592 ts_manager.cc:194] Registered new tserver with Master: 0986c1d29f814cae913e5491624c982d (127.0.147.67:40953)
I20260809 12:06:56.376318 20592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:57285
I20260809 12:06:56.380964 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:06:56.398453 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.385752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.0.147.126:38449)
0809 12:06:56.385964 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.385972 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.386417 (+ 445us) server_negotiation.cc:407] Connection header received
0809 12:06:56.386541 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.386545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.386616 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.386693 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:56.388324 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.389048 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.390350 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.390749 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.390852 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.393316 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:56.393321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:56.393324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:56.393353 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:56.396094 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:56.396731 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:56.396735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:56.396737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:56.396848 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:56.397210 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:56.397213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:56.397215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:56.397505 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:56.397696 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.397967 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.398134 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:06:56.402403 20592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:06:56.409468 20592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:06:56.449353 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.421563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36701 (local address 127.0.0.1:45418)
0809 12:06:56.441169 (+ 19606us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:56.441193 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:56.441334 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:56.442108 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:56.442113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:56.442143 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:56.442626 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.442639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.444297 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.444303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.446489 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.446502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.448095 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.448102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.448404 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.448453 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 12:06:56.448619 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 12:06:56.448847 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19431,"mutex_wait_us":1,"spinlock_wait_cycles":7424,"thread_start_us":19210,"threads_started":1}
I20260809 12:06:56.450009 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.424388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60382)
0809 12:06:56.442097 (+ 17709us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:56.442113 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:56.442344 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:56.443263 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:56.443273 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:56.443302 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:56.443785 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.443800 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.445214 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.445220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.446810 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.446825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.448983 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.448988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.449105 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.449225 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 12:06:56.449340 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 12:06:56.449467 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17574,"mutex_wait_us":1,"spinlock_wait_cycles":5632,"thread_start_us":13249,"threads_started":1}
I20260809 12:06:56.450142 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.422214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.0.147.66:36701)
0809 12:06:56.441470 (+ 19256us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.441479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.441519 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:06:56.441618 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.441624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.441849 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.442060 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:56.442841 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.444148 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.446669 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.447875 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.448036 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.448328 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.448863 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.449040 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19079,"thread_start_us":19205,"threads_started":1}
I20260809 12:06:56.450212 21303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.424758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.0.147.65:36953)
0809 12:06:56.439134 (+ 14376us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.439143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.442218 (+ 3075us) server_negotiation.cc:407] Connection header received
0809 12:06:56.442579 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.442588 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.442801 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.442971 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:56.443973 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.445057 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.447038 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.448537 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.448660 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.448863 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.449381 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.449722 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14202,"thread_start_us":14021,"threads_started":1}
I20260809 12:06:56.450583 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.421393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:40953 (local address 127.0.0.1:57692)
0809 12:06:56.438143 (+ 16750us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:56.438209 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:56.438435 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:56.439987 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:56.439998 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:56.440057 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:56.440584 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.440601 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.443159 (+ 2558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.443167 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.445753 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.445773 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.448643 (+ 2870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.448649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.449324 (+ 675us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.449367 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 12:06:56.449451 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 12:06:56.449595 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16420,"mutex_wait_us":463,"thread_start_us":20436,"threads_started":1}
I20260809 12:06:56.451099 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.421684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57692 (local address 127.0.147.67:40953)
0809 12:06:56.437588 (+ 15904us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.437600 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.438328 (+ 728us) server_negotiation.cc:407] Connection header received
0809 12:06:56.438689 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.438701 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.439287 (+ 586us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.439560 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:56.440851 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.443021 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.446792 (+ 3771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.448416 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.448569 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.448806 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.449974 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.450168 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15709,"thread_start_us":15551,"threads_started":1}
I20260809 12:06:56.454231 20780 tablet_service.cc:1467] Processing CreateTablet for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 (DEFAULT_TABLE table=test-table [id=1fe41cc67653486daea938c7152dc3d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:06:56.454360 20977 tablet_service.cc:1467] Processing CreateTablet for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 (DEFAULT_TABLE table=test-table [id=1fe41cc67653486daea938c7152dc3d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:06:56.454499 21176 tablet_service.cc:1467] Processing CreateTablet for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 (DEFAULT_TABLE table=test-table [id=1fe41cc67653486daea938c7152dc3d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:06:56.455617 20780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:56.455770 20977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:56.455770 21176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:56.466995 21304 tablet_bootstrap.cc:492] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: Bootstrap starting.
I20260809 12:06:56.470330 21305 tablet_bootstrap.cc:492] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Bootstrap starting.
I20260809 12:06:56.470382 21304 tablet_bootstrap.cc:654] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:56.471204 21308 tablet_bootstrap.cc:492] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: Bootstrap starting.
I20260809 12:06:56.471772 21304 log.cc:824] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:56.474627 21308 tablet_bootstrap.cc:654] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:56.474694 21305 tablet_bootstrap.cc:654] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:56.475096 21304 tablet_bootstrap.cc:492] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: No bootstrap required, opened a new log
I20260809 12:06:56.475380 21304 ts_tablet_manager.cc:1397] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 12:06:56.475754 21308 log.cc:824] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:56.475992 21305 log.cc:824] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:56.478369 21308 tablet_bootstrap.cc:492] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: No bootstrap required, opened a new log
I20260809 12:06:56.478655 21308 ts_tablet_manager.cc:1397] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260809 12:06:56.480337 21305 tablet_bootstrap.cc:492] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: No bootstrap required, opened a new log
I20260809 12:06:56.480588 21305 ts_tablet_manager.cc:1397] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260809 12:06:56.485162 21304 raft_consensus.cc:348] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } }
I20260809 12:06:56.485606 21304 raft_consensus.cc:374] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:56.485715 21304 raft_consensus.cc:729] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f032bebe61649a79978794ee4f1c1c2, State: Initialized, Role: FOLLOWER
I20260809 12:06:56.486332 21304 consensus_queue.cc:260] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } }
I20260809 12:06:56.487326 21308 raft_consensus.cc:348] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } }
I20260809 12:06:56.487814 21308 raft_consensus.cc:374] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:56.487987 21308 raft_consensus.cc:729] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0986c1d29f814cae913e5491624c982d, State: Initialized, Role: FOLLOWER
I20260809 12:06:56.488276 21304 ts_tablet_manager.cc:1428] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260809 12:06:56.488588 21308 consensus_queue.cc:260] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } }
I20260809 12:06:56.489744 21275 heartbeater.cc:503] Master 127.0.147.126:38449 was elected leader, sending a full tablet report...
I20260809 12:06:56.490471 21308 ts_tablet_manager.cc:1428] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260809 12:06:56.492087 21305 raft_consensus.cc:348] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } }
I20260809 12:06:56.492543 21305 raft_consensus.cc:374] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:56.492731 21305 raft_consensus.cc:729] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3cf7ebe58f4c50ae303d5a5e404e10, State: Initialized, Role: FOLLOWER
I20260809 12:06:56.493331 21305 consensus_queue.cc:260] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } }
I20260809 12:06:56.494483 21070 heartbeater.cc:503] Master 127.0.147.126:38449 was elected leader, sending a full tablet report...
I20260809 12:06:56.495151 21305 ts_tablet_manager.cc:1428] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
W20260809 12:06:56.516482 20883 tablet.cc:2369] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:06:56.517644 21315 raft_consensus.cc:482] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:06:56.517947 21315 raft_consensus.cc:504] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } }
I20260809 12:06:56.519506 21315 leader_election.cc:288] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0986c1d29f814cae913e5491624c982d (127.0.147.67:40953), 4f032bebe61649a79978794ee4f1c1c2 (127.0.147.65:36953)
I20260809 12:06:56.527143 21318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.519843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.147.66:41871)
0809 12:06:56.521073 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:56.521099 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:56.521281 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:56.521682 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:56.521686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:56.521708 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:56.522102 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.522119 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.523354 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.523360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.525013 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.525025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.526334 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.526340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.526513 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.526582 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 12:06:56.526682 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 12:06:56.526805 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"thread_start_us":1432,"threads_started":1}
I20260809 12:06:56.527168 21303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.519964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:41871 (local address 127.0.147.65:36953)
0809 12:06:56.520178 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.520184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.521213 (+ 1029us) server_negotiation.cc:407] Connection header received
0809 12:06:56.521455 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.521463 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.521547 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.521657 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:56.522255 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.523225 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.525186 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.526106 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.526233 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.526285 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.526724 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.526864 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:06:56.527493 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.519977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:56757 (local address 127.0.147.67:40953)
0809 12:06:56.520201 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.520209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.521776 (+ 1567us) server_negotiation.cc:407] Connection header received
0809 12:06:56.521946 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.521952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.522035 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.522160 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:56.522828 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.523780 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.525575 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.526460 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.526617 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.526704 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.527074 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.527224 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 12:06:56.527767 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.520014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:40953 (local address 127.0.147.66:56757)
0809 12:06:56.521662 (+ 1648us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:56.521680 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:56.521813 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:56.522190 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:56.522197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:56.522225 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:56.522676 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.522689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.523912 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.523916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.525421 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:56.525437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.526695 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.526701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:56.526889 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.526925 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:06:56.527020 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 12:06:56.527149 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1502,"thread_start_us":606,"threads_started":1}
I20260809 12:06:56.528373 20811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87cb9ebf7fb74a429ed18b33f3dbcaa6" candidate_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f032bebe61649a79978794ee4f1c1c2" is_pre_election: true
I20260809 12:06:56.528828 21209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87cb9ebf7fb74a429ed18b33f3dbcaa6" candidate_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0986c1d29f814cae913e5491624c982d" is_pre_election: true
I20260809 12:06:56.529029 20811 raft_consensus.cc:2454] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a3cf7ebe58f4c50ae303d5a5e404e10 in term 0.
I20260809 12:06:56.529407 21209 raft_consensus.cc:2454] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a3cf7ebe58f4c50ae303d5a5e404e10 in term 0.
I20260809 12:06:56.529785 20913 leader_election.cc:302] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3cf7ebe58f4c50ae303d5a5e404e10, 4f032bebe61649a79978794ee4f1c1c2; no voters:
I20260809 12:06:56.530242 21315 raft_consensus.cc:2781] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:06:56.530382 21315 raft_consensus.cc:482] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:06:56.530486 21315 raft_consensus.cc:3037] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:56.533020 21315 raft_consensus.cc:504] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } }
I20260809 12:06:56.533772 21315 leader_election.cc:288] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [CANDIDATE]: Term 1 election: Requested vote from peers 0986c1d29f814cae913e5491624c982d (127.0.147.67:40953), 4f032bebe61649a79978794ee4f1c1c2 (127.0.147.65:36953)
I20260809 12:06:56.534188 21209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87cb9ebf7fb74a429ed18b33f3dbcaa6" candidate_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0986c1d29f814cae913e5491624c982d"
I20260809 12:06:56.534315 20811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87cb9ebf7fb74a429ed18b33f3dbcaa6" candidate_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f032bebe61649a79978794ee4f1c1c2"
I20260809 12:06:56.534415 21209 raft_consensus.cc:3037] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:56.534541 20811 raft_consensus.cc:3037] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:56.537878 21209 raft_consensus.cc:2454] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a3cf7ebe58f4c50ae303d5a5e404e10 in term 1.
I20260809 12:06:56.537871 20811 raft_consensus.cc:2454] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a3cf7ebe58f4c50ae303d5a5e404e10 in term 1.
I20260809 12:06:56.538381 20909 leader_election.cc:302] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0986c1d29f814cae913e5491624c982d, 3a3cf7ebe58f4c50ae303d5a5e404e10; no voters:
I20260809 12:06:56.538797 21315 raft_consensus.cc:2781] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:06:56.539285 21315 raft_consensus.cc:686] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 LEADER]: Becoming Leader. State: Replica: 3a3cf7ebe58f4c50ae303d5a5e404e10, State: Running, Role: LEADER
I20260809 12:06:56.539857 21315 consensus_queue.cc:237] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } }
I20260809 12:06:56.544988 20592 catalog_manager.cc:5495] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a3cf7ebe58f4c50ae303d5a5e404e10 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } health_report { overall_health: UNKNOWN } } }
W20260809 12:06:56.560853 21071 tablet.cc:2369] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:06:56.569142 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.563515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.0.147.66:36701)
0809 12:06:56.563756 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.563762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.563781 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:06:56.564014 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.564018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.564100 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.564258 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:06:56.564643 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.565398 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.566844 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.567261 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.567399 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.568020 (+ 621us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:06:56.568522 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:06:56.568718 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.568771 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.568857 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:06:56.577739 21209 raft_consensus.cc:1261] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [term 1 FOLLOWER]: Refusing update from remote peer 3a3cf7ebe58f4c50ae303d5a5e404e10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:06:56.577773 20811 raft_consensus.cc:1261] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [term 1 FOLLOWER]: Refusing update from remote peer 3a3cf7ebe58f4c50ae303d5a5e404e10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:06:56.578816 21319 consensus_queue.cc:1035] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:06:56.579178 21315 consensus_queue.cc:1035] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:06:56.602584 21276 tablet.cc:2369] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:06:56.608544 20592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:06:56.608870 20592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:06:56.614768 20592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Sending DeleteTablet for 3 replicas of tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6
I20260809 12:06:56.615937 21176 tablet_service.cc:1514] Processing DeleteTablet for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:56 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:57692
I20260809 12:06:56.617252 20977 tablet_service.cc:1514] Processing DeleteTablet for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:56 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:45418
I20260809 12:06:56.621170 20780 tablet_service.cc:1514] Processing DeleteTablet for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:56 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:60382
I20260809 12:06:56.621843 20976 tablet_service.cc:1514] Processing DeleteTablet for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:56 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:45418
I20260809 12:06:56.622195 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 20687
I20260809 12:06:56.630668 21340 tablet_replica.cc:330] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: stopping tablet replica
I20260809 12:06:56.632990 21337 tablet_replica.cc:330] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: stopping tablet replica
I20260809 12:06:56.633423 21337 raft_consensus.cc:2229] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:06:56.633951 21337 raft_consensus.cc:2258] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:06:56.635593 21340 raft_consensus.cc:2229] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:06:56.636740 21340 raft_consensus.cc:2258] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:06:56.642247 21337 ts_tablet_manager.cc:1905] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:06:56.645550 21340 ts_tablet_manager.cc:1905] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:06:56.649084 21339 tablet_replica.cc:330] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: stopping tablet replica
I20260809 12:06:56.649917 21337 ts_tablet_manager.cc:1918] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:06:56.650192 21337 log.cc:1196] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/87cb9ebf7fb74a429ed18b33f3dbcaa6
I20260809 12:06:56.650873 21337 ts_tablet_manager.cc:1939] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 0986c1d29f814cae913e5491624c982d: Deleting consensus metadata
I20260809 12:06:56.649541 21339 raft_consensus.cc:2229] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:06:56.652314 21339 raft_consensus.cc:2258] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:06:56.653052 20566 catalog_manager.cc:4841] TS 0986c1d29f814cae913e5491624c982d (127.0.147.67:40953): tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 (table test-table [id=1fe41cc67653486daea938c7152dc3d0]) successfully deleted
I20260809 12:06:56.654361 21340 ts_tablet_manager.cc:1918] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:06:56.656883 21340 log.cc:1196] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/87cb9ebf7fb74a429ed18b33f3dbcaa6
I20260809 12:06:56.656953 21339 ts_tablet_manager.cc:1905] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:06:56.657572 21340 ts_tablet_manager.cc:1939] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 4f032bebe61649a79978794ee4f1c1c2: Deleting consensus metadata
I20260809 12:06:56.659488 20570 catalog_manager.cc:4841] TS 4f032bebe61649a79978794ee4f1c1c2 (127.0.147.65:36953): tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 (table test-table [id=1fe41cc67653486daea938c7152dc3d0]) successfully deleted
I20260809 12:06:56.663172 21339 ts_tablet_manager.cc:1918] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:06:56.663373 21339 log.cc:1196] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/87cb9ebf7fb74a429ed18b33f3dbcaa6
I20260809 12:06:56.664052 21339 ts_tablet_manager.cc:1939] T 87cb9ebf7fb74a429ed18b33f3dbcaa6 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Deleting consensus metadata
Aug 09 12:06:56 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:06:56.665990 20568 catalog_manager.cc:4841] TS 3a3cf7ebe58f4c50ae303d5a5e404e10 (127.0.147.66:36701): tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 (table test-table [id=1fe41cc67653486daea938c7152dc3d0]) successfully deleted
W20260809 12:06:56.666682 20568 catalog_manager.cc:4808] TS 3a3cf7ebe58f4c50ae303d5a5e404e10 (127.0.147.66:36701): delete failed for tablet 87cb9ebf7fb74a429ed18b33f3dbcaa6 because the tablet was not found. No further retry: Not found: Tablet not found: 87cb9ebf7fb74a429ed18b33f3dbcaa6
I20260809 12:06:56.672951 21303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.623854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60396 (local address 127.0.147.65:36953)
0809 12:06:56.624030 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.624038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.648128 (+ 24090us) server_negotiation.cc:407] Connection header received
0809 12:06:56.649752 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.649758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.649844 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.649917 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:56.655716 (+ 5799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.656791 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.658898 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.659624 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.659730 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.665709 (+ 5979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:56.665726 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:56.665744 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:56.665791 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:56.669714 (+ 3923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:56.670457 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:56.670468 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:56.670472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:56.670660 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:56.671131 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:56.671137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:56.671140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:56.671699 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:56.672020 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.672296 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.672530 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 12:06:56.673692 20875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:56.827778 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 20687
I20260809 12:06:56.867110 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 20889
Aug 09 12:06:56 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:06:56.891034 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:56.868049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.0.147.66:36701)
0809 12:06:56.868258 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:56.868263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:56.868354 (+ 91us) server_negotiation.cc:407] Connection header received
0809 12:06:56.868549 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:56.868554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:56.868618 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:56.868737 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:56.870469 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.871329 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.873329 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:56.873773 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:56.873870 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:56.884520 (+ 10650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:56.884538 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:56.884554 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:56.884593 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:56.888019 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:56.888776 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:56.888783 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:56.888786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:56.888932 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:56.889373 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:56.889379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:56.889383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:56.889768 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:56.890123 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:56.890408 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 12:06:56.890639 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:06:56.891863 21063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:57.009220 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 20889
I20260809 12:06:57.035709 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 21078
Aug 09 12:06:57 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:06:57.050938 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:57.036539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57708 (local address 127.0.147.67:40953)
0809 12:06:57.037022 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:57.037027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:57.037044 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:06:57.037102 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:57.037106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:57.037172 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:57.037264 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:57.038464 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:57.039162 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:57.040625 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:57.041014 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:57.041106 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:57.045490 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:57.045504 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:57.045530 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:57.045570 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:57.048299 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:57.049068 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:57.049075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:57.049077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:57.049223 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:57.049607 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:57.049614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:57.049619 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:57.049996 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:57.050211 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:57.050494 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 12:06:57.050665 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":256,"threads_started":1}
I20260809 12:06:57.051669 21266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:57.142100 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 21078
I20260809 12:06:57.164608 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 20541
I20260809 12:06:57.165766 20640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:57.277120 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 20541
I20260809 12:06:57.307194 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38449
--webserver_interface=127.0.147.126
--webserver_port=33733
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:38449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:06:57.647341 21349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:57.647701 21349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:57.647861 21349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:57.656385 21349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:06:57.656519 21349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:57.656605 21349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:57.656701 21349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:06:57.656771 21349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:06:57.666358 21349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:38449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=33733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:57.667690 21349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:57.668974 21349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:57.695710 21357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:57.695803 21358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:57.695807 21361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:57.696450 21349 server_base.cc:1034] running on GCE node
I20260809 12:06:57.697464 21349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:57.699123 21349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:57.700374 21349 hybrid_clock.cc:648] HybridClock initialized: now 1786277217700315 us; error 53 us; skew 500 ppm
I20260809 12:06:57.700820 21349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:57 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277217, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:57.708612 21349 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:06:57.710449 21349 webserver.cc:460] Webserver started at http://127.0.147.126:33733/ using document root <none> and password file <none>
I20260809 12:06:57.711227 21349 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:57.711413 21349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:57.717586 21349 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 12:06:57.720655 21369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:57.722007 21349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:06:57.722211 21349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "a94b6cb3467e46a59938bd85d4e0a8d1"
format_stamp: "Formatted at 2026-08-09 12:06:54 on dist-test-slave-sb2z"
server_key: "5e2ac90aaab6ce0a9a9ac6e1f119f7e6"
server_key_iv: "dec5db35effd3818ea46cda0a5946376"
server_key_version: "encryptionkey@0"
I20260809 12:06:57.723306 21349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:57.755617 21349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:57.760628 21349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:57.760941 21349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:57.781204 21349 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:38449
I20260809 12:06:57.781231 21448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:38449 every 8 connection(s)
I20260809 12:06:57.782850 21349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:06:57.789497 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 21349
I20260809 12:06:57.790385 21449 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:06:57.790593 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:36953
--local_ip_for_outbound_sockets=127.0.147.65
--tserver_master_addrs=127.0.147.126:38449
--webserver_port=44595
--webserver_interface=127.0.147.65
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:06:57.795706 21449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Bootstrap starting.
I20260809 12:06:57.817988 21449 log.cc:824] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:57.830485 21449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:06:57.831053 21449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Bootstrap complete.
I20260809 12:06:57.840370 21449 raft_consensus.cc:348] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:06:57.841243 21449 raft_consensus.cc:729] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94b6cb3467e46a59938bd85d4e0a8d1, State: Initialized, Role: FOLLOWER
I20260809 12:06:57.841871 21449 consensus_queue.cc:260] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:06:57.842110 21449 raft_consensus.cc:388] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:06:57.842238 21449 raft_consensus.cc:482] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:06:57.842442 21449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:06:57.846329 21449 raft_consensus.cc:504] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:06:57.846855 21449 leader_election.cc:302] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a94b6cb3467e46a59938bd85d4e0a8d1; no voters:
I20260809 12:06:57.847489 21449 leader_election.cc:288] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:06:57.847658 21454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:06:57.849211 21454 raft_consensus.cc:686] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 2 LEADER]: Becoming Leader. State: Replica: a94b6cb3467e46a59938bd85d4e0a8d1, State: Running, Role: LEADER
I20260809 12:06:57.849810 21454 consensus_queue.cc:237] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:06:57.850059 21449 sys_catalog.cc:564] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:06:57.852634 21460 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } } }
I20260809 12:06:57.853114 21460 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:57.855568 21461 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94b6cb3467e46a59938bd85d4e0a8d1. Latest consensus state: current_term: 2 leader_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } } }
I20260809 12:06:57.855924 21461 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:06:57.857378 21465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:06:57.865213 21465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:06:57.866631 21465 catalog_manager.cc:1259] Loaded cluster ID: 8aa1c7b4b56d477d886a3f456e933348
I20260809 12:06:57.866773 21465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:06:57.870059 21465 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 12:06:57.888389 21465 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 12:06:57.909837 21465 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:06:57.911751 21465 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Loaded TSK: 0
I20260809 12:06:57.913301 21465 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:06:58.168148 21452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:58.168417 21452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:58.168568 21452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:58.175741 21452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:58.175853 21452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:58.176050 21452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:06:58.187222 21452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:36953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=44595
--tserver_master_addrs=127.0.147.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:58.188625 21452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:58.190073 21452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:58.205560 21492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:58.205834 21490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:58.206472 21452 server_base.cc:1034] running on GCE node
W20260809 12:06:58.206532 21494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:58.207222 21452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:58.208433 21452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:58.209682 21452 hybrid_clock.cc:648] HybridClock initialized: now 1786277218209616 us; error 58 us; skew 500 ppm
I20260809 12:06:58.210095 21452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:58 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277218, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:58.213950 21452 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:06:58.216029 21452 webserver.cc:460] Webserver started at http://127.0.147.65:44595/ using document root <none> and password file <none>
I20260809 12:06:58.216782 21452 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:58.216979 21452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:58.222939 21452 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 12:06:58.225991 21506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:58.227734 21452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:06:58.227931 21452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "4f032bebe61649a79978794ee4f1c1c2"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "68852ec8a895eb21bacb0c8a2fad971d"
server_key_iv: "36e27e5a9d2e4f497b8799d9f6e3de44"
server_key_version: "encryptionkey@0"
I20260809 12:06:58.229081 21452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:58.273473 21452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:58.278447 21452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:58.278764 21452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:58.280126 21452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:58.281817 21452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:58.281970 21452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:58.282222 21452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:58.282342 21452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:58.320849 21452 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:36953
I20260809 12:06:58.320958 21687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:36953 every 8 connection(s)
I20260809 12:06:58.322358 21452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:06:58.323982 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 21452
I20260809 12:06:58.324718 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:36701
--local_ip_for_outbound_sockets=127.0.147.66
--tserver_master_addrs=127.0.147.126:38449
--webserver_port=46157
--webserver_interface=127.0.147.66
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:06:58 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277218, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:58.348168 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:58.325802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:46243)
0809 12:06:58.326746 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:58.326790 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:58.328615 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:58.330209 (+ 1594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:58.330216 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:58.330758 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:58.331316 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:58.331334 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:58.333677 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:58.333681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:58.335025 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:58.335039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:58.336466 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:58.336471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:58.336662 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:58.337369 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:58.337403 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:58.340195 (+ 2792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:58.343816 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:58.343824 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:58.343837 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:58.344300 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:58.344766 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:58.344771 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:58.344775 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:58.344959 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:58.345694 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:58.345702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:58.346021 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 12:06:58.346293 (+ 272us) client_negotiation.cc:240] Negotiation successful
0809 12:06:58.346812 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":426,"threads_started":1}
I20260809 12:06:58.348165 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:58.325515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:46243 (local address 127.0.147.126:38449)
0809 12:06:58.326221 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:58.326238 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:58.326930 (+ 692us) server_negotiation.cc:407] Connection header received
0809 12:06:58.329228 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:58.329248 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:58.329686 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:58.330080 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:58.331583 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:58.333504 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:58.335191 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:58.336247 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:58.336458 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:58.340455 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:58.340462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:58.340475 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:58.340520 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:58.343659 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:58.344438 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:58.344448 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:58.344459 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:58.344623 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:58.345091 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:58.345096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:58.345098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:58.345536 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:58.345734 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:58.346534 (+ 800us) server_negotiation.cc:299] Negotiation successful
0809 12:06:58.346930 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":280,"threads_started":1}
I20260809 12:06:58.351380 21692 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38449
I20260809 12:06:58.351727 21692 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:58.352445 21692 heartbeater.cc:511] Master 127.0.147.126:38449 requested a full tablet report, sending...
I20260809 12:06:58.354969 21400 ts_manager.cc:194] Registered new tserver with Master: 4f032bebe61649a79978794ee4f1c1c2 (127.0.147.65:36953)
I20260809 12:06:58.357332 21400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:46243
W20260809 12:06:58.668864 21700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:58.669127 21700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:58.669222 21700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:58.676384 21700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:58.676489 21700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:58.676615 21700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:06:58.686172 21700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:36701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=46157
--tserver_master_addrs=127.0.147.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:58.687402 21700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:58.688692 21700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:58.701629 21707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:58.701725 21708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:58.701749 21711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:58.702788 21700 server_base.cc:1034] running on GCE node
I20260809 12:06:58.703384 21700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:58.704326 21700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:58.705564 21700 hybrid_clock.cc:648] HybridClock initialized: now 1786277218705512 us; error 58 us; skew 500 ppm
I20260809 12:06:58.705871 21700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:58 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277218, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:58.709120 21700 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:06:58.710683 21700 webserver.cc:460] Webserver started at http://127.0.147.66:46157/ using document root <none> and password file <none>
I20260809 12:06:58.711351 21700 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:58.711515 21700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:58.716975 21700 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 12:06:58.719493 21723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:58.720871 21700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:06:58.721063 21700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "bcf3812d69105a1bcb0d8606ba6a1de3"
server_key_iv: "3fdd12260240e7a5fe47d766eb79b997"
server_key_version: "encryptionkey@0"
I20260809 12:06:58.722036 21700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:58.760293 21700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:58.764520 21700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:58.764813 21700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:58.766005 21700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:58.767494 21700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:58.767684 21700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:58.767813 21700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:58.767912 21700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:58.803833 21700 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:36701
I20260809 12:06:58.803918 21892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:36701 every 8 connection(s)
I20260809 12:06:58.805208 21700 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:06:58.807266 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 21700
I20260809 12:06:58.808027 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:40953
--local_ip_for_outbound_sockets=127.0.147.67
--tserver_master_addrs=127.0.147.126:38449
--webserver_port=37755
--webserver_interface=127.0.147.67
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:06:58 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277218, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:58.826758 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:58.807954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:60731 (local address 127.0.147.126:38449)
0809 12:06:58.808177 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:58.808183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:58.809593 (+ 1410us) server_negotiation.cc:407] Connection header received
0809 12:06:58.811488 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:58.811496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:58.811712 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:58.811847 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:58.813454 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:58.814554 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:58.816162 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:58.816882 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:58.817035 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:58.820603 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:58.820608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:58.820611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:58.820647 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:58.823161 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:58.824113 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:58.824124 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:58.824126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:58.824250 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:58.824775 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:58.824778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:58.824780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:58.825024 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:58.825144 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:58.826161 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0809 12:06:58.826421 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:06:58.827953 21897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:58.808483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:60731)
0809 12:06:58.809393 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:58.809437 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:58.811222 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:58.812060 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:58.812068 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:58.812648 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:58.813214 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:58.813231 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:58.814728 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:58.814731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:58.816013 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:58.816026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:58.817089 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:58.817094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:58.817279 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:58.817951 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:58.817972 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:58.820366 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:58.823298 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:58.823309 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:58.823324 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:58.823978 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:58.824377 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:58.824385 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:58.824391 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:58.824645 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:58.825156 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:58.825166 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:58.825611 (+ 445us) client_negotiation.cc:769] Sending connection context
0809 12:06:58.825986 (+ 375us) client_negotiation.cc:240] Negotiation successful
0809 12:06:58.826548 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":401,"threads_started":1}
I20260809 12:06:58.829442 21894 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38449
I20260809 12:06:58.829788 21894 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:58.830494 21894 heartbeater.cc:511] Master 127.0.147.126:38449 requested a full tablet report, sending...
I20260809 12:06:58.831957 21400 ts_manager.cc:194] Registered new tserver with Master: 3a3cf7ebe58f4c50ae303d5a5e404e10 (127.0.147.66:36701)
I20260809 12:06:58.832820 21400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:60731
W20260809 12:06:59.152729 21899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:06:59.152993 21899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:06:59.153198 21899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:06:59.160511 21899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:06:59.160624 21899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:06:59.160838 21899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:06:59.170311 21899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:40953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=37755
--tserver_master_addrs=127.0.147.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:06:59.171597 21899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:06:59.172847 21899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:06:59.186295 21905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:59.185990 21910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:06:59.186090 21907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:06:59.187274 21899 server_base.cc:1034] running on GCE node
I20260809 12:06:59.187906 21899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:06:59.188822 21899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:06:59.190063 21899 hybrid_clock.cc:648] HybridClock initialized: now 1786277219190009 us; error 43 us; skew 500 ppm
I20260809 12:06:59.190376 21899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:06:59 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277219, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:06:59.193580 21899 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:06:59.195022 21899 webserver.cc:460] Webserver started at http://127.0.147.67:37755/ using document root <none> and password file <none>
I20260809 12:06:59.195778 21899 fs_manager.cc:359] Metadata directory not provided
I20260809 12:06:59.195946 21899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:06:59.201241 21899 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:06:59.203897 21922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:59.205207 21899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:06:59.205392 21899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "0986c1d29f814cae913e5491624c982d"
format_stamp: "Formatted at 2026-08-09 12:06:56 on dist-test-slave-sb2z"
server_key: "3d13094249bf09c787d993b60db1c5eb"
server_key_iv: "437be60854ece156a43947fb11417430"
server_key_version: "encryptionkey@0"
I20260809 12:06:59.206393 21899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:06:59.219847 21899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:06:59.224341 21899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:06:59.224625 21899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:06:59.225708 21899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:06:59.227136 21899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:06:59.227257 21899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:59.227386 21899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:06:59.227474 21899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:06:59.262077 21899 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:40953
I20260809 12:06:59.262137 22084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:40953 every 8 connection(s)
I20260809 12:06:59.263582 21899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:06:59.272229 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 21899
Aug 09 12:06:59 dist-test-slave-sb2z krb5kdc[20520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277219, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:06:59.285823 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.266089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:49489 (local address 127.0.147.126:38449)
0809 12:06:59.266260 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.266264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.267467 (+ 1203us) server_negotiation.cc:407] Connection header received
0809 12:06:59.269284 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.269288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.269352 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.269439 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:59.270830 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.271586 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.273395 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.274362 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.274491 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.279181 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:59.279186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:59.279189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:59.279220 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:59.282777 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.283468 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.283476 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.283482 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.283663 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.284180 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.284188 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.284192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.284550 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:59.284711 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.285397 (+ 686us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.285570 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:06:59.287215 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.266596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:49489)
0809 12:06:59.267312 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.267349 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:59.269034 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:59.269628 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:59.269635 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:59.270156 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:06:59.270682 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.270698 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.271713 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.271717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.273202 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.273215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.274948 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.274958 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.275365 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.276254 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:06:59.276284 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:06:59.278961 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:06:59.282916 (+ 3955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:59.282922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:59.282932 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:59.283345 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:59.283800 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:06:59.283803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:06:59.283805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:06:59.284059 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:06:59.284699 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:06:59.284705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:06:59.284985 (+ 280us) client_negotiation.cc:769] Sending connection context
0809 12:06:59.285278 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 12:06:59.285811 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":295,"threads_started":1}
I20260809 12:06:59.288693 22086 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38449
I20260809 12:06:59.289001 22086 heartbeater.cc:464] Registering TS with master...
I20260809 12:06:59.289716 22086 heartbeater.cc:511] Master 127.0.147.126:38449 requested a full tablet report, sending...
I20260809 12:06:59.291146 21400 ts_manager.cc:194] Registered new tserver with Master: 0986c1d29f814cae913e5491624c982d (127.0.147.67:40953)
I20260809 12:06:59.291584 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.273162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.0.147.126:38449)
0809 12:06:59.273635 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.273639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.276128 (+ 2489us) server_negotiation.cc:407] Connection header received
0809 12:06:59.276290 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.276295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.276372 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.276489 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:59.278207 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.279276 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.281385 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.281856 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.281949 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.285297 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:59.285305 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:59.285310 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:59.285345 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:59.288593 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.289359 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.289366 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.289369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.289491 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.289934 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.289938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.289940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.290191 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:59.290455 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.290787 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.291010 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":255,"threads_started":1}
I20260809 12:06:59.292323 21400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:49489
I20260809 12:06:59.294068 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:06:59.300263 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.294963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.0.147.126:38449)
0809 12:06:59.295125 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.295129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.295704 (+ 575us) server_negotiation.cc:407] Connection header received
0809 12:06:59.295846 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.295849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.295966 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.296094 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:06:59.296441 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.297090 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.298479 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.298851 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.298937 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.299384 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:06:59.299849 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:06:59.299951 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.300045 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.300105 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:06:59.301107 21400 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:06:59.308307 21400 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:06:59.341110 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.321043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:40953 (local address 127.0.0.1:57722)
0809 12:06:59.331848 (+ 10805us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.331870 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:59.332017 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:59.332801 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:59.332811 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:59.332867 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:59.333399 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.333415 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.335671 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.335678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.338209 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.338249 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.340255 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.340268 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.340457 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.340494 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 12:06:59.340593 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 12:06:59.340720 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10646,"thread_start_us":11338,"threads_started":1}
I20260809 12:06:59.341200 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.324409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57722 (local address 127.0.147.67:40953)
0809 12:06:59.329920 (+ 5511us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.329931 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.332000 (+ 2069us) server_negotiation.cc:407] Connection header received
0809 12:06:59.332182 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.332192 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.332493 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.333677 (+ 1184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:59.334035 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.335461 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.338435 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.339793 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.340062 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.340292 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.340659 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.340833 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5323,"thread_start_us":6116,"threads_started":1}
I20260809 12:06:59.341574 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.324774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45452 (local address 127.0.147.66:36701)
0809 12:06:59.330391 (+ 5617us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.330398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.332914 (+ 2516us) server_negotiation.cc:407] Connection header received
0809 12:06:59.333015 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.333023 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.333311 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.333569 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:59.334245 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.336200 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.338305 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.339781 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.340838 (+ 1057us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.341053 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.341118 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.341285 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5445,"thread_start_us":5517,"threads_started":1}
I20260809 12:06:59.341825 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.321616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:36701 (local address 127.0.0.1:45452)
0809 12:06:59.332662 (+ 11046us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.332680 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:59.332794 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:59.333736 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:59.333740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:59.333758 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:59.334041 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.334048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.336381 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.336387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.338118 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.338134 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.340049 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.340056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.340243 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.340303 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 12:06:59.340413 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 12:06:59.340542 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10924,"mutex_wait_us":2,"thread_start_us":9088,"threads_started":1}
I20260809 12:06:59.342737 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.320998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60398)
0809 12:06:59.330999 (+ 10001us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.331038 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:59.331226 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:59.335295 (+ 4069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:59.335300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:59.335325 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:59.335911 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.335926 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.337900 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.337906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.339646 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.339659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.342019 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.342026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.342200 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.342235 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:06:59.342322 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 12:06:59.342439 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9714,"thread_start_us":10500,"threads_started":1}
I20260809 12:06:59.342902 22100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.322797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.0.147.65:36953)
0809 12:06:59.334787 (+ 11990us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.334795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.334820 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:06:59.334901 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.334908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.335097 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.335281 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:59.336130 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.337749 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.339828 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.341457 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.341598 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.341830 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.342413 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.342578 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11830,"thread_start_us":11661,"threads_started":1}
I20260809 12:06:59.345449 21996 tablet_service.cc:1467] Processing CreateTablet for tablet 86022723498642a7b461f2708a7d1203 (DEFAULT_TABLE table=test-table [id=872a8d90576641ca8df3217d2dc2e3f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:06:59.345718 21791 tablet_service.cc:1467] Processing CreateTablet for tablet 86022723498642a7b461f2708a7d1203 (DEFAULT_TABLE table=test-table [id=872a8d90576641ca8df3217d2dc2e3f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:06:59.346737 21996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86022723498642a7b461f2708a7d1203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:59.346310 21575 tablet_service.cc:1467] Processing CreateTablet for tablet 86022723498642a7b461f2708a7d1203 (DEFAULT_TABLE table=test-table [id=872a8d90576641ca8df3217d2dc2e3f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:06:59.347060 21791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86022723498642a7b461f2708a7d1203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:59.347719 21575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86022723498642a7b461f2708a7d1203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:06:59.358934 22108 tablet_bootstrap.cc:492] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: Bootstrap starting.
I20260809 12:06:59.360657 21692 heartbeater.cc:503] Master 127.0.147.126:38449 was elected leader, sending a full tablet report...
I20260809 12:06:59.363763 22108 tablet_bootstrap.cc:654] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:59.364883 22108 log.cc:824] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:59.367857 22108 tablet_bootstrap.cc:492] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: No bootstrap required, opened a new log
I20260809 12:06:59.368136 22108 ts_tablet_manager.cc:1397] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 12:06:59.368257 22109 tablet_bootstrap.cc:492] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Bootstrap starting.
I20260809 12:06:59.368775 22110 tablet_bootstrap.cc:492] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Bootstrap starting.
I20260809 12:06:59.371400 22110 tablet_bootstrap.cc:654] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:59.371958 22109 tablet_bootstrap.cc:654] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Neither blocks nor log segments found. Creating new log.
I20260809 12:06:59.372925 22110 log.cc:824] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:59.373167 22109 log.cc:824] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Log is configured to *not* fsync() on all Append() calls
I20260809 12:06:59.375758 22109 tablet_bootstrap.cc:492] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: No bootstrap required, opened a new log
I20260809 12:06:59.376045 22109 ts_tablet_manager.cc:1397] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260809 12:06:59.376063 22110 tablet_bootstrap.cc:492] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: No bootstrap required, opened a new log
I20260809 12:06:59.376324 22110 ts_tablet_manager.cc:1397] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260809 12:06:59.378526 22108 raft_consensus.cc:348] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } }
I20260809 12:06:59.378978 22108 raft_consensus.cc:374] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:59.379076 22108 raft_consensus.cc:729] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0986c1d29f814cae913e5491624c982d, State: Initialized, Role: FOLLOWER
I20260809 12:06:59.379747 22108 consensus_queue.cc:260] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } }
I20260809 12:06:59.381134 22086 heartbeater.cc:503] Master 127.0.147.126:38449 was elected leader, sending a full tablet report...
I20260809 12:06:59.381770 22108 ts_tablet_manager.cc:1428] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 12:06:59.384814 22109 raft_consensus.cc:348] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } }
I20260809 12:06:59.385175 22109 raft_consensus.cc:374] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:59.385303 22109 raft_consensus.cc:729] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3cf7ebe58f4c50ae303d5a5e404e10, State: Initialized, Role: FOLLOWER
I20260809 12:06:59.385892 22109 consensus_queue.cc:260] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } }
I20260809 12:06:59.386642 22110 raft_consensus.cc:348] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } }
I20260809 12:06:59.386998 21894 heartbeater.cc:503] Master 127.0.147.126:38449 was elected leader, sending a full tablet report...
I20260809 12:06:59.387094 22110 raft_consensus.cc:374] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:06:59.387197 22110 raft_consensus.cc:729] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f032bebe61649a79978794ee4f1c1c2, State: Initialized, Role: FOLLOWER
I20260809 12:06:59.387679 22109 ts_tablet_manager.cc:1428] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260809 12:06:59.387905 22110 consensus_queue.cc:260] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } }
I20260809 12:06:59.389477 22110 ts_tablet_manager.cc:1428] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260809 12:06:59.515327 22087 tablet.cc:2369] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:06:59.521552 22116 raft_consensus.cc:482] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:06:59.521898 22116 raft_consensus.cc:504] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } }
I20260809 12:06:59.524056 22116 leader_election.cc:288] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f032bebe61649a79978794ee4f1c1c2 (127.0.147.65:36953), 0986c1d29f814cae913e5491624c982d (127.0.147.67:40953)
I20260809 12:06:59.531500 22100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.524448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:38755 (local address 127.0.147.65:36953)
0809 12:06:59.524765 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.524773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.525703 (+ 930us) server_negotiation.cc:407] Connection header received
0809 12:06:59.525951 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.525956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.526039 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.526151 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:59.526896 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.527925 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.529752 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.530532 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.530731 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.530811 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.531104 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.531247 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260809 12:06:59.531800 22119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.524570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.147.66:38755)
0809 12:06:59.525554 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.525579 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:59.525772 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:59.526204 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:59.526208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:59.526248 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:59.526733 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.526745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.528058 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.528072 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.529586 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.529598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.530760 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.530766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.530922 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.530945 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:06:59.531207 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 12:06:59.531332 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":1,"spinlock_wait_cycles":22912,"thread_start_us":1134,"threads_started":1}
I20260809 12:06:59.531895 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.524459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:41155 (local address 127.0.147.67:40953)
0809 12:06:59.524683 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.524689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.526222 (+ 1533us) server_negotiation.cc:407] Connection header received
0809 12:06:59.526559 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.526565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.526648 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.526748 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:06:59.527412 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.528284 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.530016 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.530809 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.530995 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.531075 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.531466 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.531640 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:06:59.532172 22118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.524570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:40953 (local address 127.0.147.66:41155)
0809 12:06:59.526107 (+ 1537us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.526127 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:06:59.526352 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:06:59.526757 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:06:59.526761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:06:59.526780 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:06:59.527235 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.527248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.528423 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.528428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.529851 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:06:59.529863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.531049 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.531055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:06:59.531307 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.531333 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 12:06:59.531442 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 12:06:59.531585 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1382,"mutex_wait_us":83,"thread_start_us":471,"threads_started":1}
I20260809 12:06:59.533116 21612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86022723498642a7b461f2708a7d1203" candidate_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f032bebe61649a79978794ee4f1c1c2" is_pre_election: true
I20260809 12:06:59.533249 22022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86022723498642a7b461f2708a7d1203" candidate_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0986c1d29f814cae913e5491624c982d" is_pre_election: true
I20260809 12:06:59.533654 21612 raft_consensus.cc:2454] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a3cf7ebe58f4c50ae303d5a5e404e10 in term 0.
I20260809 12:06:59.533882 22022 raft_consensus.cc:2454] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a3cf7ebe58f4c50ae303d5a5e404e10 in term 0.
I20260809 12:06:59.534468 21730 leader_election.cc:302] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3cf7ebe58f4c50ae303d5a5e404e10, 4f032bebe61649a79978794ee4f1c1c2; no voters:
I20260809 12:06:59.534894 22116 raft_consensus.cc:2781] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:06:59.535048 22116 raft_consensus.cc:482] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:06:59.535171 22116 raft_consensus.cc:3037] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:59.537381 22116 raft_consensus.cc:504] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } }
I20260809 12:06:59.537997 22116 leader_election.cc:288] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [CANDIDATE]: Term 1 election: Requested vote from peers 4f032bebe61649a79978794ee4f1c1c2 (127.0.147.65:36953), 0986c1d29f814cae913e5491624c982d (127.0.147.67:40953)
I20260809 12:06:59.538362 21612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86022723498642a7b461f2708a7d1203" candidate_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f032bebe61649a79978794ee4f1c1c2"
I20260809 12:06:59.538524 22022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86022723498642a7b461f2708a7d1203" candidate_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0986c1d29f814cae913e5491624c982d"
I20260809 12:06:59.538546 21612 raft_consensus.cc:3037] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:59.538738 22022 raft_consensus.cc:3037] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:06:59.540850 21612 raft_consensus.cc:2454] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a3cf7ebe58f4c50ae303d5a5e404e10 in term 1.
I20260809 12:06:59.541128 22022 raft_consensus.cc:2454] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a3cf7ebe58f4c50ae303d5a5e404e10 in term 1.
I20260809 12:06:59.541271 21730 leader_election.cc:302] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3cf7ebe58f4c50ae303d5a5e404e10, 4f032bebe61649a79978794ee4f1c1c2; no voters:
I20260809 12:06:59.541535 22116 raft_consensus.cc:2781] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:06:59.542011 22116 raft_consensus.cc:686] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 LEADER]: Becoming Leader. State: Replica: 3a3cf7ebe58f4c50ae303d5a5e404e10, State: Running, Role: LEADER
I20260809 12:06:59.542476 22116 consensus_queue.cc:237] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } }
I20260809 12:06:59.546773 21400 catalog_manager.cc:5495] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a3cf7ebe58f4c50ae303d5a5e404e10 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 36701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 } health_report { overall_health: UNKNOWN } } }
W20260809 12:06:59.557252 21895 tablet.cc:2369] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:06:59.574857 21693 tablet.cc:2369] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:06:59.580794 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.575373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.0.147.66:36701)
0809 12:06:59.575675 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.575681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.575708 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:06:59.575868 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.575873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.575974 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.576109 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:06:59.576459 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.577193 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.578575 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.579012 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.579110 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.579628 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:06:59.580150 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:06:59.580262 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.580363 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.580464 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 12:06:59.588972 21612 raft_consensus.cc:1261] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [term 1 FOLLOWER]: Refusing update from remote peer 3a3cf7ebe58f4c50ae303d5a5e404e10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:06:59.588979 22022 raft_consensus.cc:1261] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [term 1 FOLLOWER]: Refusing update from remote peer 3a3cf7ebe58f4c50ae303d5a5e404e10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:06:59.590126 22120 consensus_queue.cc:1035] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0986c1d29f814cae913e5491624c982d" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 40953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:06:59.590477 22116 consensus_queue.cc:1035] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f032bebe61649a79978794ee4f1c1c2" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 36953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:06:59.603250 22128 mvcc.cc:205] Tried to move back new op lower bound from 7316591491425796096 to 7316591491249594368. Current Snapshot: MvccSnapshot[applied={T|T < 7316591491425796096}]
I20260809 12:06:59.617357 21400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:06:59.617635 21400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:06:59.621199 21400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Sending DeleteTablet for 3 replicas of tablet 86022723498642a7b461f2708a7d1203
I20260809 12:06:59.622399 21575 tablet_service.cc:1514] Processing DeleteTablet for tablet 86022723498642a7b461f2708a7d1203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:59 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:60398
I20260809 12:06:59.622599 21996 tablet_service.cc:1514] Processing DeleteTablet for tablet 86022723498642a7b461f2708a7d1203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:59 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:57722
I20260809 12:06:59.622931 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 21452
I20260809 12:06:59.623701 21791 tablet_service.cc:1514] Processing DeleteTablet for tablet 86022723498642a7b461f2708a7d1203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:06:59 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:45452
I20260809 12:06:59.635146 22142 tablet_replica.cc:330] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: stopping tablet replica
I20260809 12:06:59.635766 22142 raft_consensus.cc:2229] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:06:59.636415 22142 raft_consensus.cc:2258] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:06:59.638803 22142 ts_tablet_manager.cc:1905] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:06:59.642587 22100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.623823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.0.147.65:36953)
0809 12:06:59.624343 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.624352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.624377 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:06:59.624447 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.624454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.624553 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.624687 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:59.626558 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.628154 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.630204 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.631148 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.631279 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.634870 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:59.634891 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:59.634910 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:59.634958 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:59.638959 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.640062 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.640071 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.640074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.640253 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.640735 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.640739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.640742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.641226 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:59.641533 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.641921 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.642173 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352}
I20260809 12:06:59.643756 21683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:59.646801 22143 tablet_replica.cc:330] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: stopping tablet replica
I20260809 12:06:59.647373 22143 raft_consensus.cc:2229] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:06:59.648063 22143 raft_consensus.cc:2258] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:06:59.648391 22144 tablet_replica.cc:330] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: stopping tablet replica
I20260809 12:06:59.650463 22144 raft_consensus.cc:2229] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:06:59.650784 22143 ts_tablet_manager.cc:1905] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:06:59.652060 22144 raft_consensus.cc:2258] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:06:59.653812 22144 ts_tablet_manager.cc:1905] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:06:59.658114 22143 ts_tablet_manager.cc:1918] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:06:59.658285 22143 log.cc:1196] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/86022723498642a7b461f2708a7d1203
I20260809 12:06:59.658869 22143 ts_tablet_manager.cc:1939] T 86022723498642a7b461f2708a7d1203 P 0986c1d29f814cae913e5491624c982d: Deleting consensus metadata
I20260809 12:06:59.661724 22144 ts_tablet_manager.cc:1918] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:06:59.661921 22144 log.cc:1196] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/86022723498642a7b461f2708a7d1203
I20260809 12:06:59.662559 22144 ts_tablet_manager.cc:1939] T 86022723498642a7b461f2708a7d1203 P 3a3cf7ebe58f4c50ae303d5a5e404e10: Deleting consensus metadata
I20260809 12:06:59.664218 21372 catalog_manager.cc:4841] TS 0986c1d29f814cae913e5491624c982d (127.0.147.67:40953): tablet 86022723498642a7b461f2708a7d1203 (table test-table [id=872a8d90576641ca8df3217d2dc2e3f3]) successfully deleted
I20260809 12:06:59.664510 21375 catalog_manager.cc:4841] TS 3a3cf7ebe58f4c50ae303d5a5e404e10 (127.0.147.66:36701): tablet 86022723498642a7b461f2708a7d1203 (table test-table [id=872a8d90576641ca8df3217d2dc2e3f3]) successfully deleted
I20260809 12:06:59.743512 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 21452
W20260809 12:06:59.769677 21377 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.147.65:36953 (error 108)
I20260809 12:06:59.770025 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 21700
I20260809 12:06:59.772097 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.771520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60412)
0809 12:06:59.771777 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.771913 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260809 12:06:59.772480 21377 catalog_manager.cc:4584] TS 4f032bebe61649a79978794ee4f1c1c2 (127.0.147.65:36953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86022723498642a7b461f2708a7d1203: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36953: connect: Connection refused (error 111)
I20260809 12:06:59.789165 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.771820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45474 (local address 127.0.147.66:36701)
0809 12:06:59.772001 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.772010 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.772028 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:06:59.772086 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.772090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.772150 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.772275 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:59.774151 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.775001 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.776955 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.777390 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.777484 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.781378 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:59.781395 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:59.781415 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:59.781464 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:59.785553 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.786471 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.786481 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.786486 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.786666 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.787120 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.787125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.787129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.787680 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:59.787986 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.788491 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.788737 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:06:59.790056 21888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:59.826201 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.825591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60418)
0809 12:06:59.825864 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.826040 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260809 12:06:59.827298 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.826943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60428)
0809 12:06:59.827111 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.827187 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 12:06:59.887809 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.887264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60442)
0809 12:06:59.887490 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.887650 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 12:06:59.888880 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.888535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60450)
0809 12:06:59.888689 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.888764 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 12:06:59.908362 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 21700
I20260809 12:06:59.936308 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 21899
I20260809 12:06:59.949291 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.937191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.0.147.67:40953)
0809 12:06:59.937412 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:06:59.937417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:06:59.937437 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:06:59.937617 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:06:59.937621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:06:59.937685 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:06:59.937767 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:06:59.939136 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.939923 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.941236 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:06:59.941603 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:06:59.941691 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:06:59.944196 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:06:59.944211 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:06:59.944227 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:06:59.944261 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:06:59.946769 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.947404 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.947411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.947413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.947619 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:06:59.947957 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:06:59.947960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:06:59.947962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:06:59.948296 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:06:59.948516 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 12:06:59.948786 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 12:06:59.948937 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 12:06:59.950013 22080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:06:59.969630 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.969156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60466)
0809 12:06:59.969349 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.969445 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 12:06:59.970726 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:06:59.970340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60478)
0809 12:06:59.970544 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 12:06:59.970617 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 12:07:00.040367 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 21899
I20260809 12:07:00.065816 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 21349
I20260809 12:07:00.067073 21442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:00.183806 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:00.182975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:36953 (local address 127.0.0.1:60482)
0809 12:07:00.183257 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:00.183395 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 12:07:00.184216 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 21349
I20260809 12:07:00.214704 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38449
--webserver_interface=127.0.147.126
--webserver_port=33733
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:38449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:07:00.555121 22150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:00.555404 22150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:00.555507 22150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:00.562703 22150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:07:00.562812 22150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:00.562868 22150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:00.562927 22150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:07:00.562975 22150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:07:00.572520 22150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:38449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=33733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:00.573721 22150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:00.574911 22150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:00.586750 22158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:00.586797 22161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:00.586818 22159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:00.588049 22150 server_base.cc:1034] running on GCE node
I20260809 12:07:00.588917 22150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:00.590276 22150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:00.591559 22150 hybrid_clock.cc:648] HybridClock initialized: now 1786277220591472 us; error 71 us; skew 500 ppm
I20260809 12:07:00.591931 22150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:00 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277220, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:00.595108 22150 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:07:00.596601 22150 webserver.cc:460] Webserver started at http://127.0.147.126:33733/ using document root <none> and password file <none>
I20260809 12:07:00.597303 22150 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:00.597458 22150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:00.602849 22150 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:07:00.605640 22171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:00.606925 22150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:07:00.607117 22150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "a94b6cb3467e46a59938bd85d4e0a8d1"
format_stamp: "Formatted at 2026-08-09 12:06:54 on dist-test-slave-sb2z"
server_key: "5e2ac90aaab6ce0a9a9ac6e1f119f7e6"
server_key_iv: "dec5db35effd3818ea46cda0a5946376"
server_key_version: "encryptionkey@0"
I20260809 12:07:00.608158 22150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:00.619371 22150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:00.623169 22150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:00.623425 22150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:00.642031 22150 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:38449
I20260809 12:07:00.642078 22252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:38449 every 8 connection(s)
I20260809 12:07:00.643498 22150 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:07:00.650133 22253 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:07:00.653256 22253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Bootstrap starting.
I20260809 12:07:00.653956 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 22150
I20260809 12:07:00.654803 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:36953
--local_ip_for_outbound_sockets=127.0.147.65
--tserver_master_addrs=127.0.147.126:38449
--webserver_port=44595
--webserver_interface=127.0.147.65
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:07:00.674958 22253 log.cc:824] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Log is configured to *not* fsync() on all Append() calls
I20260809 12:07:00.691622 22253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:07:00.692183 22253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1: Bootstrap complete.
I20260809 12:07:00.701339 22253 raft_consensus.cc:348] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:07:00.702209 22253 raft_consensus.cc:729] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94b6cb3467e46a59938bd85d4e0a8d1, State: Initialized, Role: FOLLOWER
I20260809 12:07:00.702797 22253 consensus_queue.cc:260] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:07:00.703035 22253 raft_consensus.cc:388] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:07:00.703176 22253 raft_consensus.cc:482] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:07:00.703361 22253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260809 12:07:00.707019 22253 raft_consensus.cc:504] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:07:00.707531 22253 leader_election.cc:302] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a94b6cb3467e46a59938bd85d4e0a8d1; no voters:
I20260809 12:07:00.708283 22253 leader_election.cc:288] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 12:07:00.708418 22259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 12:07:00.709739 22259 raft_consensus.cc:686] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [term 3 LEADER]: Becoming Leader. State: Replica: a94b6cb3467e46a59938bd85d4e0a8d1, State: Running, Role: LEADER
I20260809 12:07:00.710247 22259 consensus_queue.cc:237] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } }
I20260809 12:07:00.710518 22253 sys_catalog.cc:564] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:07:00.713052 22260 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } } }
I20260809 12:07:00.713541 22260 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:07:00.713044 22261 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94b6cb3467e46a59938bd85d4e0a8d1. Latest consensus state: current_term: 3 leader_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94b6cb3467e46a59938bd85d4e0a8d1" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38449 } } }
I20260809 12:07:00.714097 22261 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94b6cb3467e46a59938bd85d4e0a8d1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:07:00.716248 22267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:07:00.724866 22267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:07:00.726370 22267 catalog_manager.cc:1259] Loaded cluster ID: 8aa1c7b4b56d477d886a3f456e933348
I20260809 12:07:00.726526 22267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 12:07:00.728745 22267 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6e06a9c9ad google::LogMessage::Fail() at ??:0
@ 0x7f6e06a9f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6e06a9c4de google::LogMessage::Flush() at ??:0
@ 0x7f6e06a9da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6e1c4cf512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6e1c4cc976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6e1c554610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f6e1c554439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6e096e8345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6e096eb91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6e096eb799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6e096c6eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6e0c9a06db start_thread at ??:0
@ 0x7f6e0593661f clone at ??:0
W20260809 12:07:01.037328 22256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:01.037642 22256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:01.037812 22256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:01.045215 22256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:01.045367 22256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:01.045542 22256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:07:01.055179 22256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:36953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=44595
--tserver_master_addrs=127.0.147.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:01.056538 22256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:01.057891 22256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:01.071069 22318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:01.071373 22313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:01.071168 22323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:01.072263 22256 server_base.cc:1034] running on GCE node
I20260809 12:07:01.072865 22256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:01.073786 22256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:01.075021 22256 hybrid_clock.cc:648] HybridClock initialized: now 1786277221074975 us; error 66 us; skew 500 ppm
I20260809 12:07:01.075332 22256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:01 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277221, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:01.078624 22256 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:07:01.080209 22256 webserver.cc:460] Webserver started at http://127.0.147.65:44595/ using document root <none> and password file <none>
I20260809 12:07:01.080878 22256 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:01.081084 22256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:01.086699 22256 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 12:07:01.090046 22333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:01.091444 22256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:07:01.091652 22256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "4f032bebe61649a79978794ee4f1c1c2"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "68852ec8a895eb21bacb0c8a2fad971d"
server_key_iv: "36e27e5a9d2e4f497b8799d9f6e3de44"
server_key_version: "encryptionkey@0"
I20260809 12:07:01.092707 22256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:01.124881 22256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:01.129132 22256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:01.129437 22256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:01.130498 22256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:07:01.132470 22344 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260809 12:07:01.137962 22344 ts_tablet_manager.cc:1852] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 12:07:01.138934 22344 ts_tablet_manager.cc:1905] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:07:01.146028 22344 ts_tablet_manager.cc:1918] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:07:01.146236 22344 log.cc:1196] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/86022723498642a7b461f2708a7d1203
I20260809 12:07:01.146801 22344 ts_tablet_manager.cc:1939] T 86022723498642a7b461f2708a7d1203 P 4f032bebe61649a79978794ee4f1c1c2: Deleting consensus metadata
I20260809 12:07:01.147250 22256 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 12:07:01.147403 22256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260809 12:07:01.147563 22256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:07:01.147661 22256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:01.183269 22256 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:36953
I20260809 12:07:01.183327 22508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:36953 every 8 connection(s)
I20260809 12:07:01.184595 22256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:07:01.187786 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 22256
I20260809 12:07:01.188474 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:36701
--local_ip_for_outbound_sockets=127.0.147.66
--tserver_master_addrs=127.0.147.126:38449
--webserver_port=46157
--webserver_interface=127.0.147.66
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:07:01.189742 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:01.187631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:37983)
0809 12:07:01.188566 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:01.188868 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":387,"threads_started":1}
W20260809 12:07:01.190374 22509 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:38449 (0 consecutive failures): Network error: Failed to ping master at 127.0.147.126:38449: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
I20260809 12:07:01.191296 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:01.190952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:48231)
0809 12:07:01.191133 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:01.191200 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 12:07:01.192170 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:01.191846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:34815)
0809 12:07:01.191983 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:01.192062 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:07:01.192411 22509 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:07:01.539253 22512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:01.539592 22512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:01.539716 22512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:01.546862 22512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:01.546972 22512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:01.547092 22512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:07:01.557473 22512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:36701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=46157
--tserver_master_addrs=127.0.147.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:01.558722 22512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:01.560128 22512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:01.572986 22519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:01.573031 22524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:01.573122 22521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:01.574021 22512 server_base.cc:1034] running on GCE node
I20260809 12:07:01.574644 22512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:01.575866 22512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:01.577100 22512 hybrid_clock.cc:648] HybridClock initialized: now 1786277221577061 us; error 49 us; skew 500 ppm
I20260809 12:07:01.577422 22512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:01 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277221, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:01.580721 22512 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:07:01.582247 22512 webserver.cc:460] Webserver started at http://127.0.147.66:46157/ using document root <none> and password file <none>
I20260809 12:07:01.582890 22512 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:01.583035 22512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:01.588143 22512 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:07:01.590744 22536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:01.592137 22512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:07:01.592306 22512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "3a3cf7ebe58f4c50ae303d5a5e404e10"
format_stamp: "Formatted at 2026-08-09 12:06:55 on dist-test-slave-sb2z"
server_key: "bcf3812d69105a1bcb0d8606ba6a1de3"
server_key_iv: "3fdd12260240e7a5fe47d766eb79b997"
server_key_version: "encryptionkey@0"
I20260809 12:07:01.593277 22512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:01.623365 22512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:01.627331 22512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:01.627625 22512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:01.628600 22512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:07:01.629905 22512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:07:01.630029 22512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:01.630153 22512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:07:01.630249 22512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:01.664290 22512 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:36701
I20260809 12:07:01.664381 22696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:36701 every 8 connection(s)
I20260809 12:07:01.665728 22512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:07:01.671842 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 22512
I20260809 12:07:01.672351 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:01.669149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:38165)
0809 12:07:01.670906 (+ 1757us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:01.671228 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1452,"thread_start_us":1251,"threads_started":1}
I20260809 12:07:01.672529 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:40953
--local_ip_for_outbound_sockets=127.0.147.67
--tserver_master_addrs=127.0.147.126:38449
--webserver_port=37755
--webserver_interface=127.0.147.67
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:07:01.677712 22698 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:38449 (0 consecutive failures): Network error: Failed to ping master at 127.0.147.126:38449: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
I20260809 12:07:01.678808 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:01.678363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:39655)
0809 12:07:01.678580 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:01.678661 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260809 12:07:01.679764 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:01.679366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:54869)
0809 12:07:01.679566 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:01.679655 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 12:07:01.680013 22698 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:07:02.019083 22701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:02.019371 22701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:02.019467 22701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:02.026894 22701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:02.027006 22701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:02.027117 22701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:07:02.037001 22701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:40953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=37755
--tserver_master_addrs=127.0.147.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:02.038239 22701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:02.039541 22701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:02.052467 22707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:02.052515 22710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:02.052446 22708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:02.053625 22701 server_base.cc:1034] running on GCE node
I20260809 12:07:02.054240 22701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:02.055189 22701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:02.056424 22701 hybrid_clock.cc:648] HybridClock initialized: now 1786277222056400 us; error 137 us; skew 500 ppm
I20260809 12:07:02.056748 22701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:02 dist-test-slave-sb2z krb5kdc[20520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277222, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:02.060025 22701 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:07:02.061560 22701 webserver.cc:460] Webserver started at http://127.0.147.67:37755/ using document root <none> and password file <none>
I20260809 12:07:02.062240 22701 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:02.062391 22701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:02.067780 22701 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:07:02.070370 22721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:02.071671 22701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:07:02.071847 22701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "0986c1d29f814cae913e5491624c982d"
format_stamp: "Formatted at 2026-08-09 12:06:56 on dist-test-slave-sb2z"
server_key: "3d13094249bf09c787d993b60db1c5eb"
server_key_iv: "437be60854ece156a43947fb11417430"
server_key_version: "encryptionkey@0"
I20260809 12:07:02.072809 22701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:02.093648 22701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:02.098112 22701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:02.098416 22701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:02.099395 22701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:07:02.100769 22701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:07:02.100857 22701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:02.101030 22701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:07:02.101140 22701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:02.135519 22701 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:40953
I20260809 12:07:02.135690 22882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:40953 every 8 connection(s)
I20260809 12:07:02.137115 22701 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:07:02.141826 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:02.140016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:58123)
0809 12:07:02.140729 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:02.140991 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":258,"threads_started":1}
W20260809 12:07:02.142475 22884 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:38449 (0 consecutive failures): Network error: Failed to ping master at 127.0.147.126:38449: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
I20260809 12:07:02.143280 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:02.142951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:33363)
0809 12:07:02.143088 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:02.143149 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 12:07:02.144130 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:02.143804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:38917)
0809 12:07:02.143917 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:02.143986 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 12:07:02.144480 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 22701
W20260809 12:07:02.144730 22884 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 12:07:02.193616 22890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:02.192876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:52167)
0809 12:07:02.193276 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:02.193395 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":207,"threads_started":1}
I20260809 12:07:02.681581 22893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:02.680656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:38031)
0809 12:07:02.681239 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:02.681331 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":314,"threads_started":1}
I20260809 12:07:03.145925 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:03.145192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:57955)
0809 12:07:03.145616 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:03.145702 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":206,"threads_started":1}
I20260809 12:07:03.195205 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:03.194457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:42725)
0809 12:07:03.194870 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:03.194993 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":237,"threads_started":1}
I20260809 12:07:03.683585 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:03.682602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:34845)
0809 12:07:03.683179 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:03.683310 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":294,"threads_started":1}
I20260809 12:07:04.147534 22901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:04.146787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:35015)
0809 12:07:04.147272 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:04.147350 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":255,"threads_started":1}
I20260809 12:07:04.196843 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:04.196047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:52893)
0809 12:07:04.196587 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:04.196672 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":311,"threads_started":1}
I20260809 12:07:04.685352 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:04.684396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:39495)
0809 12:07:04.685023 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:04.685132 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":314,"threads_started":1}
I20260809 12:07:05.149063 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:05.148326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:48851)
0809 12:07:05.148817 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:05.148895 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":304,"threads_started":1}
I20260809 12:07:05.198498 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:05.197726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:49199)
0809 12:07:05.198226 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:05.198317 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":275,"threads_started":1}
I20260809 12:07:05.687016 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:05.686305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:57673)
0809 12:07:05.686756 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:05.686836 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":235,"threads_started":1}
I20260809 12:07:06.150709 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:06.149949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:39335)
0809 12:07:06.150411 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:06.150489 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":247,"threads_started":1}
I20260809 12:07:06.200404 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:06.199368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:48123)
0809 12:07:06.200148 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:06.200241 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":624,"threads_started":1}
I20260809 12:07:06.688658 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:06.687931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:52789)
0809 12:07:06.688384 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:06.688465 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":239,"threads_started":1}
I20260809 12:07:07.152628 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:07.151716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:48511)
0809 12:07:07.152245 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:07.152375 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":255,"threads_started":1}
I20260809 12:07:07.201987 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:07.201305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:35301)
0809 12:07:07.201751 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:07.201831 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":236,"threads_started":1}
I20260809 12:07:07.690440 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:07.689575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:35809)
0809 12:07:07.690057 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:07.690180 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":222,"threads_started":1}
I20260809 12:07:08.154323 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:08.153627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:55463)
0809 12:07:08.154070 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:08.154163 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":221,"threads_started":1}
I20260809 12:07:08.203715 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:08.202859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:40967)
0809 12:07:08.203349 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:08.203428 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":236,"threads_started":1}
I20260809 12:07:08.692660 22922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:08.691792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:36361)
0809 12:07:08.692354 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:08.692439 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":286,"threads_started":1}
I20260809 12:07:09.156073 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:09.155237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:36273)
0809 12:07:09.155782 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:09.155860 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":204,"threads_started":1}
I20260809 12:07:09.205197 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:09.204519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:37329)
0809 12:07:09.204933 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:09.205037 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":222,"threads_started":1}
I20260809 12:07:09.694226 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:09.693537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:55529)
0809 12:07:09.693975 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:09.694056 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
I20260809 12:07:10.157732 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:10.157012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:54299)
0809 12:07:10.157462 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:10.157566 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":236,"threads_started":1}
I20260809 12:07:10.206753 22930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:10.206037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:59559)
0809 12:07:10.206471 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:10.206579 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":212,"threads_started":1}
I20260809 12:07:10.696421 22931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:10.695311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:51747)
0809 12:07:10.695965 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:10.696143 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":353,"threads_started":1}
I20260809 12:07:11.159389 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:11.158647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:55041)
0809 12:07:11.159118 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:11.159201 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":229,"threads_started":1}
I20260809 12:07:11.208451 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:11.207757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:54375)
0809 12:07:11.208211 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:11.208291 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":238,"threads_started":1}
I20260809 12:07:11.698177 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:11.697346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:52827)
0809 12:07:11.697807 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:11.697930 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":251,"threads_started":1}
I20260809 12:07:12.161098 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:12.160351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:39133)
0809 12:07:12.160814 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:12.160893 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":276,"threads_started":1}
I20260809 12:07:12.210268 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:12.209396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:38135)
0809 12:07:12.209990 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:12.210083 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":323,"threads_started":1}
I20260809 12:07:12.699756 22939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:12.698991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:52665)
0809 12:07:12.699420 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:12.699509 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":205,"threads_started":1}
I20260809 12:07:13.162770 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:13.161926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:54221)
0809 12:07:13.162484 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:13.162592 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":375,"threads_started":1}
I20260809 12:07:13.211961 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:13.211205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:38177)
0809 12:07:13.211710 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:13.211798 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":230,"threads_started":1}
I20260809 12:07:13.701370 22943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:13.700668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:47745)
0809 12:07:13.701129 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:13.701211 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":245,"threads_started":1}
I20260809 12:07:14.164551 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:14.163827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:52647)
0809 12:07:14.164296 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:14.164395 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":209,"threads_started":1}
I20260809 12:07:14.213577 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:14.212800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:60195)
0809 12:07:14.213255 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:14.213371 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":223,"threads_started":1}
I20260809 12:07:14.703032 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:14.702255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:38985)
0809 12:07:14.702738 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:14.702832 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":239,"threads_started":1}
I20260809 12:07:15.166235 22951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:15.165418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:39115)
0809 12:07:15.165903 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:15.165995 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":262,"threads_started":1}
I20260809 12:07:15.215171 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:15.214443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:34831)
0809 12:07:15.214904 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:15.214990 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":233,"threads_started":1}
I20260809 12:07:15.704655 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:15.703954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:51233)
0809 12:07:15.704387 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:15.704471 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":224,"threads_started":1}
I20260809 12:07:16.168087 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:16.167243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:34665)
0809 12:07:16.167779 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:16.167865 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":326,"threads_started":1}
I20260809 12:07:16.217007 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:16.216260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.65:50953)
0809 12:07:16.216733 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:16.216821 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":228,"threads_started":1}
I20260809 12:07:16.706888 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:16.706010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.66:57769)
0809 12:07:16.706570 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:16.706667 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":300,"threads_started":1}
I20260809 12:07:17.148530 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 22256
I20260809 12:07:17.166831 22961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:17.149612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.0.147.65:36953)
0809 12:07:17.150277 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:17.150297 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:17.150343 (+ 46us) server_negotiation.cc:407] Connection header received
0809 12:07:17.152836 (+ 2493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:17.152853 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:17.153350 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:17.153653 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:17.155071 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:17.156366 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:17.157560 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:17.158219 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:17.158352 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:17.161046 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:17.161053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:17.161067 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:17.161107 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:17.163982 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:17.164647 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:17.164653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:17.164659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:17.164807 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:17.165174 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:17.165177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:17.165179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:17.165666 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:17.165885 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:17.166222 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 12:07:17.166527 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":331,"threads_started":1}
I20260809 12:07:17.168272 22503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:17.176339 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:17.169423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38449 (local address 127.0.147.67:39347)
0809 12:07:17.176005 (+ 6582us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:17.176147 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6413,"thread_start_us":6304,"threads_started":1}
I20260809 12:07:17.260322 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 22256
I20260809 12:07:17.282665 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 22512
I20260809 12:07:17.300784 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:17.283685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.0.147.66:36701)
0809 12:07:17.284218 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:17.284231 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:17.284271 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:07:17.285834 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:17.285844 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:17.286279 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:17.286637 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:17.288274 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:17.289499 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:17.290770 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:17.291439 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:17.291665 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:17.294380 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:17.294387 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:17.294402 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:17.294438 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:17.297449 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:17.298209 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:17.298216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:17.298221 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:17.298386 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:17.298782 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:17.298788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:17.298791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:17.299280 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:17.299628 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:17.300106 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 12:07:17.300498 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":242,"threads_started":1}
I20260809 12:07:17.302104 22691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:17.395062 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 22512
I20260809 12:07:17.419502 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 22701
I20260809 12:07:17.438400 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:17.420556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.0.147.67:40953)
0809 12:07:17.421142 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:17.421157 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:17.421200 (+ 43us) server_negotiation.cc:407] Connection header received
0809 12:07:17.422887 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:17.422896 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:17.423333 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:17.423777 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:17.425432 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:17.426678 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:17.428123 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:17.428800 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:17.429029 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:17.432007 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:17.432015 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:17.432030 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:17.432080 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:17.435012 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:17.436075 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:17.436081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:17.436087 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:17.436256 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:17.436583 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:17.436586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:17.436588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:17.437105 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:17.437339 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:17.437667 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 12:07:17.438099 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":247,"threads_started":1}
I20260809 12:07:17.439845 22878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:17.524250 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 22701
2026-08-09T12:07:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24989 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:07:17 dist-test-slave-sb2z krb5kdc[22975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:07:17 dist-test-slave-sb2z krb5kdc[22975](info): set up 2 sockets
Aug 09 12:07:17 dist-test-slave-sb2z krb5kdc[22975](info): commencing operation
krb5kdc: starting...
W20260809 12:07:18.834298 589 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.003s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:07:19 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:07:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:07:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:07:19.143239 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:43625
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:43625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:07:19.489006 22991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:19.489289 22991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:19.489475 22991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:19.496608 22991 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 12:07:19.496696 22991 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 12:07:19.496788 22991 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 12:07:19.496878 22991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:07:19.496973 22991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:19.497087 22991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:19.497169 22991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:07:19.497246 22991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:07:19.507205 22991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:43625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:43625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:19.508524 22991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:19.509881 22991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:19.521971 23001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:19.522030 22997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:19.522235 22996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:19.523228 22991 server_base.cc:1034] running on GCE node
I20260809 12:07:19.524040 22991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:19.525327 22991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:19.526558 22991 hybrid_clock.cc:648] HybridClock initialized: now 1786277239526494 us; error 35 us; skew 500 ppm
I20260809 12:07:19.526906 22991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:19 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:19.530110 22991 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:07:19.531648 22991 webserver.cc:460] Webserver started at http://127.0.147.126:43635/ using document root <none> and password file <none>
I20260809 12:07:19.532387 22991 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:19.532532 22991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:19.532853 22991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:07:19.535980 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8"
format_stamp: "Formatted at 2026-08-09 12:07:19 on dist-test-slave-sb2z"
server_key: "d6e08ed0a8817f31249c40a4d42f0014"
server_key_iv: "e20538a5b23e4b4f036430e125f11291"
server_key_version: "encryptionkey@0"
I20260809 12:07:19.536798 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8"
format_stamp: "Formatted at 2026-08-09 12:07:19 on dist-test-slave-sb2z"
server_key: "d6e08ed0a8817f31249c40a4d42f0014"
server_key_iv: "e20538a5b23e4b4f036430e125f11291"
server_key_version: "encryptionkey@0"
I20260809 12:07:19.542024 22991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:07:19.545523 23013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:19.546792 22991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:07:19.546979 22991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8"
format_stamp: "Formatted at 2026-08-09 12:07:19 on dist-test-slave-sb2z"
server_key: "d6e08ed0a8817f31249c40a4d42f0014"
server_key_iv: "e20538a5b23e4b4f036430e125f11291"
server_key_version: "encryptionkey@0"
I20260809 12:07:19.547207 22991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:19.564013 22991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:19.567936 22991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:19.568215 22991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:19.587167 22991 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:43625
I20260809 12:07:19.587179 23088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:43625 every 8 connection(s)
I20260809 12:07:19.588546 22991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:07:19.592209 23090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:07:19.593308 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 22991
I20260809 12:07:19.593544 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:07:19.594296 589 external_mini_cluster.cc:1428] Setting key fccaa4fa82ab551b0eb66a8efe052a3e
I20260809 12:07:19.606706 23090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Bootstrap starting.
Aug 09 12:07:19 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:07:19.609923 23090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Neither blocks nor log segments found. Creating new log.
I20260809 12:07:19.611223 23090 log.cc:824] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Log is configured to *not* fsync() on all Append() calls
I20260809 12:07:19.614104 23090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: No bootstrap required, opened a new log
I20260809 12:07:19.618033 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:19.596417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45382 (local address 127.0.147.126:43625)
0809 12:07:19.597252 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:19.597269 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:19.597325 (+ 56us) server_negotiation.cc:407] Connection header received
0809 12:07:19.599076 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:19.599110 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:19.599611 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:19.600064 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:19.601465 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:19.603713 (+ 2248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:19.605036 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:19.606036 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:19.606216 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:19.610497 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:19.610507 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:19.610525 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:19.610567 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:19.613897 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:19.614675 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:19.614682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:19.614688 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:19.614856 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:19.615270 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:19.615275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:19.615278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:19.615880 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:19.616206 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:19.616665 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 12:07:19.617070 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":580,"thread_start_us":351,"threads_started":1}
I20260809 12:07:19.622534 23090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:07:19.622895 23090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:07:19.623021 23090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a45a9d2ff94c2c8f84a9151d88a6f8, State: Initialized, Role: FOLLOWER
I20260809 12:07:19.623646 23090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:07:19.623895 23090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:07:19.624044 23090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:07:19.624212 23090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:07:19.626439 23090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:07:19.627015 23090 leader_election.cc:302] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32a45a9d2ff94c2c8f84a9151d88a6f8; no voters:
I20260809 12:07:19.627725 23090 leader_election.cc:288] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:07:19.627846 23095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:07:19.628552 23095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 1 LEADER]: Becoming Leader. State: Replica: 32a45a9d2ff94c2c8f84a9151d88a6f8, State: Running, Role: LEADER
I20260809 12:07:19.629141 23095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:07:19.629797 23090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:07:19.631963 23101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32a45a9d2ff94c2c8f84a9151d88a6f8. Latest consensus state: current_term: 1 leader_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } } }
I20260809 12:07:19.632335 23101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:07:19.632822 23099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } } }
I20260809 12:07:19.633358 23099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:07:19.635054 23105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:07:19.641324 23105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:07:19.649801 23105 catalog_manager.cc:1347] Generated new cluster ID: 5d121e48a55b448dac144fcd384cb1d9
I20260809 12:07:19.649971 23105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:07:19.658715 23105 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:07:19.660039 23105 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:07:19.672575 23105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Generated new TSK 0
I20260809 12:07:19.673542 23105 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:07:19.753742 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:43625
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:07:20.098960 23126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:20.099325 23126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:20.099542 23126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:20.106976 23126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:20.107159 23126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:20.107352 23126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:07:20.117592 23126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:43625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:20.118947 23126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:20.120354 23126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:20.133916 23135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:20.134019 23134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:20.133878 23138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:20.135207 23126 server_base.cc:1034] running on GCE node
I20260809 12:07:20.135931 23126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:20.136993 23126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:20.138237 23126 hybrid_clock.cc:648] HybridClock initialized: now 1786277240138185 us; error 44 us; skew 500 ppm
I20260809 12:07:20.138585 23126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:20 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277240, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:20.141850 23126 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:07:20.143340 23126 webserver.cc:460] Webserver started at http://127.0.147.65:42129/ using document root <none> and password file <none>
I20260809 12:07:20.144117 23126 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:20.144272 23126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:20.144608 23126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:07:20.147666 23126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "f7cb995fc2b6456dbe4a7af289055d33"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "ccd7b233da22769135738670803fb8b8"
server_key_iv: "522c25a6f39d4807c7dd11f5a38e354d"
server_key_version: "encryptionkey@0"
I20260809 12:07:20.148470 23126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "f7cb995fc2b6456dbe4a7af289055d33"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "ccd7b233da22769135738670803fb8b8"
server_key_iv: "522c25a6f39d4807c7dd11f5a38e354d"
server_key_version: "encryptionkey@0"
I20260809 12:07:20.154340 23126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:07:20.157989 23148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:20.159310 23126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:07:20.159502 23126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "f7cb995fc2b6456dbe4a7af289055d33"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "ccd7b233da22769135738670803fb8b8"
server_key_iv: "522c25a6f39d4807c7dd11f5a38e354d"
server_key_version: "encryptionkey@0"
I20260809 12:07:20.159788 23126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:20.188834 23126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:20.193089 23126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:20.193404 23126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:20.194473 23126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:07:20.195915 23126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:07:20.196029 23126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:20.196192 23126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:07:20.196267 23126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:20.231617 23126 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:45469
I20260809 12:07:20.231685 23317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:45469 every 8 connection(s)
I20260809 12:07:20.232955 23126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:07:20.237013 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 23126
I20260809 12:07:20.237244 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:07:20.237951 589 external_mini_cluster.cc:1428] Setting key e6fd9819f0085cbb1f59ac5aaa159292
Aug 09 12:07:20 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277240, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:07:20.255442 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:20.235636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:45803 (local address 127.0.147.126:43625)
0809 12:07:20.236151 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:20.236155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:20.237594 (+ 1439us) server_negotiation.cc:407] Connection header received
0809 12:07:20.239396 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:20.239401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:20.239462 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:20.239617 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:20.241177 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:20.242367 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:20.243977 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:20.244699 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:20.244855 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:20.248489 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:20.248497 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:20.248505 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:20.248550 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:20.252252 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:20.252946 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:20.252953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:20.252959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:20.253118 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:20.253535 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:20.253543 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:20.253548 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:20.253914 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:20.254088 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:20.254762 (+ 674us) server_negotiation.cc:299] Negotiation successful
0809 12:07:20.255011 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":298,"threads_started":1}
I20260809 12:07:20.256268 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:20.236245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:45803)
0809 12:07:20.237180 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:20.237228 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:20.239094 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:20.239872 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:20.239880 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:20.240409 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:07:20.240996 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:20.241016 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:20.242529 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:20.242533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:20.243828 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:20.243840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:20.244931 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:20.244936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:20.245125 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:20.245826 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:07:20.245847 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:07:20.248262 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:07:20.252382 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:20.252388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:20.252400 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:20.252814 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:20.253226 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:20.253230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:20.253232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:20.253418 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:20.254050 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:07:20.254055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:07:20.254348 (+ 293us) client_negotiation.cc:769] Sending connection context
0809 12:07:20.254565 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 12:07:20.254931 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":364,"threads_started":1}
I20260809 12:07:20.257798 23318 heartbeater.cc:347] Connected to a master server at 127.0.147.126:43625
I20260809 12:07:20.258124 23318 heartbeater.cc:464] Registering TS with master...
I20260809 12:07:20.259153 23318 heartbeater.cc:511] Master 127.0.147.126:43625 requested a full tablet report, sending...
I20260809 12:07:20.261404 23038 ts_manager.cc:194] Registered new tserver with Master: f7cb995fc2b6456dbe4a7af289055d33 (127.0.147.65:45469)
I20260809 12:07:20.262847 23038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:45803
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:07:20.309077 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:43625
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:07:20.648237 23327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:20.648504 23327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:20.648599 23327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:20.655721 23327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:20.655820 23327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:20.655932 23327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:07:20.665680 23327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:43625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:20.666860 23327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:20.668179 23327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:20.681361 23338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:20.681452 23339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:20.681406 23342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:20.682179 23327 server_base.cc:1034] running on GCE node
I20260809 12:07:20.682780 23327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:20.683749 23327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:20.684984 23327 hybrid_clock.cc:648] HybridClock initialized: now 1786277240684937 us; error 56 us; skew 500 ppm
I20260809 12:07:20.685317 23327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:20 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277240, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:20.691278 23327 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:07:20.692801 23327 webserver.cc:460] Webserver started at http://127.0.147.66:40569/ using document root <none> and password file <none>
I20260809 12:07:20.693480 23327 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:20.693635 23327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:20.693956 23327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:07:20.696918 23327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "030d35048f184ebbacc138e11b415cf8"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "c209bfc9272956a244ebf9ff3b778685"
server_key_iv: "8740f224a76ffbeecb3a3c7fdba55ea3"
server_key_version: "encryptionkey@0"
I20260809 12:07:20.697696 23327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "030d35048f184ebbacc138e11b415cf8"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "c209bfc9272956a244ebf9ff3b778685"
server_key_iv: "8740f224a76ffbeecb3a3c7fdba55ea3"
server_key_version: "encryptionkey@0"
I20260809 12:07:20.702740 23327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:07:20.706075 23357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:20.707350 23327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:07:20.707571 23327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "030d35048f184ebbacc138e11b415cf8"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "c209bfc9272956a244ebf9ff3b778685"
server_key_iv: "8740f224a76ffbeecb3a3c7fdba55ea3"
server_key_version: "encryptionkey@0"
I20260809 12:07:20.707813 23327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:20.720815 23327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:20.725201 23327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:20.725497 23327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:20.726423 23327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:07:20.727756 23327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:07:20.727859 23327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:20.728012 23327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:07:20.728091 23327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:20.764938 23327 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:35553
I20260809 12:07:20.764977 23523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:35553 every 8 connection(s)
I20260809 12:07:20.766234 23327 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:07:20.769309 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 23327
I20260809 12:07:20.769515 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:07:20.770097 589 external_mini_cluster.cc:1428] Setting key e82395e30d037c886ec1d3d5115dacaf
Aug 09 12:07:20 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277240, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:07:20.789868 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:20.768869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:47695 (local address 127.0.147.126:43625)
0809 12:07:20.769363 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:20.769370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:20.770605 (+ 1235us) server_negotiation.cc:407] Connection header received
0809 12:07:20.772957 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:20.772962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:20.773022 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:20.773105 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:20.775263 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:20.776031 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:20.778477 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:20.778885 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:20.778987 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:20.783658 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:20.783663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:20.783667 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:20.783700 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:20.786239 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:20.787203 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:20.787210 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:20.787217 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:20.787380 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:20.787888 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:20.787896 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:20.787900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:20.788287 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:20.788467 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:20.789144 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 12:07:20.789395 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":250,"threads_started":1}
I20260809 12:07:20.790546 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:20.769479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:47695)
0809 12:07:20.770404 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:20.770460 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:20.772644 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:20.773390 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:20.773403 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:20.774229 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:07:20.775084 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:20.775113 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:20.776186 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:20.776194 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:20.778324 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:20.778342 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:20.779120 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:20.779129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:20.779476 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:20.780270 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:07:20.780303 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:07:20.783390 (+ 3087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:07:20.786393 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:20.786404 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:20.786423 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:20.787060 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:20.787502 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:20.787515 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:20.787518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:20.787772 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:20.788420 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:07:20.788426 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:07:20.788742 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 12:07:20.788977 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 12:07:20.789348 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":330,"threads_started":1}
I20260809 12:07:20.792035 23524 heartbeater.cc:347] Connected to a master server at 127.0.147.126:43625
I20260809 12:07:20.792429 23524 heartbeater.cc:464] Registering TS with master...
I20260809 12:07:20.793313 23524 heartbeater.cc:511] Master 127.0.147.126:43625 requested a full tablet report, sending...
I20260809 12:07:20.794838 23038 ts_manager.cc:194] Registered new tserver with Master: 030d35048f184ebbacc138e11b415cf8 (127.0.147.66:35553)
I20260809 12:07:20.795738 23038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:47695
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:07:20.843926 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:43625
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:07:21.189550 23531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:21.189874 23531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:21.190064 23531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:21.197090 23531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:21.197233 23531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:21.197424 23531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:07:21.206949 23531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:43625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:21.208408 23531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:21.209710 23531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:21.223266 23542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:21.223322 23538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:21.223232 23537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:21.224174 23531 server_base.cc:1034] running on GCE node
I20260809 12:07:21.224787 23531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:21.225687 23531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:21.226941 23531 hybrid_clock.cc:648] HybridClock initialized: now 1786277241226876 us; error 56 us; skew 500 ppm
I20260809 12:07:21.227248 23531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:21 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:21.230712 23531 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:07:21.232594 23531 webserver.cc:460] Webserver started at http://127.0.147.67:40529/ using document root <none> and password file <none>
I20260809 12:07:21.233403 23531 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:21.233548 23531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:21.233896 23531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:07:21.237783 23531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "cdf243980d7d40a7bbae1d7d376a0566"
format_stamp: "Formatted at 2026-08-09 12:07:21 on dist-test-slave-sb2z"
server_key: "7f6f6ceca4a44db2348ec09830262d99"
server_key_iv: "83dfd9a6d851378ea8a0d182e724efb9"
server_key_version: "encryptionkey@0"
I20260809 12:07:21.238502 23531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "cdf243980d7d40a7bbae1d7d376a0566"
format_stamp: "Formatted at 2026-08-09 12:07:21 on dist-test-slave-sb2z"
server_key: "7f6f6ceca4a44db2348ec09830262d99"
server_key_iv: "83dfd9a6d851378ea8a0d182e724efb9"
server_key_version: "encryptionkey@0"
I20260809 12:07:21.243498 23531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:07:21.246803 23551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:21.248144 23531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:07:21.248324 23531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "cdf243980d7d40a7bbae1d7d376a0566"
format_stamp: "Formatted at 2026-08-09 12:07:21 on dist-test-slave-sb2z"
server_key: "7f6f6ceca4a44db2348ec09830262d99"
server_key_iv: "83dfd9a6d851378ea8a0d182e724efb9"
server_key_version: "encryptionkey@0"
I20260809 12:07:21.248541 23531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:21.265921 23318 heartbeater.cc:503] Master 127.0.147.126:43625 was elected leader, sending a full tablet report...
I20260809 12:07:21.273533 23531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:21.278071 23531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:21.278419 23531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:21.279593 23531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:07:21.281000 23531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:07:21.281106 23531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:21.281260 23531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:07:21.281335 23531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:21.316449 23531 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:45519
I20260809 12:07:21.316506 23726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:45519 every 8 connection(s)
I20260809 12:07:21.318250 23531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:07:21.323920 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 23531
I20260809 12:07:21.324143 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:07:21.324637 589 external_mini_cluster.cc:1428] Setting key 554546c68e8e67981ea4eab21a0c07b3
Aug 09 12:07:21 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:07:21.341939 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.321990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:42155 (local address 127.0.147.126:43625)
0809 12:07:21.323444 (+ 1454us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.323453 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.323480 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:07:21.327640 (+ 4160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.327644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.327710 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.327800 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:21.329409 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.330357 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.332032 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.332416 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.332517 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.335837 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:21.335841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:21.335844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:21.335877 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:21.338708 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:21.339405 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:21.339412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:21.339417 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:21.339625 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:21.340131 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:21.340139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:21.340143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:21.340523 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:21.340662 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.341492 (+ 830us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.341664 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1310,"thread_start_us":2656,"threads_started":1}
I20260809 12:07:21.342869 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.322385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:42155)
0809 12:07:21.323294 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:21.323341 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:21.327321 (+ 3980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:21.328086 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:21.328097 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:21.328705 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:07:21.329254 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.329271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.330501 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.330504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.331889 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.331907 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.332606 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.332610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.332796 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.333394 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:07:21.333415 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:07:21.335661 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:07:21.338853 (+ 3192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:21.338859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:21.338870 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:21.339285 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:21.339774 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:21.339778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:21.339781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:21.339996 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:21.340668 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:07:21.340676 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:07:21.341139 (+ 463us) client_negotiation.cc:769] Sending connection context
0809 12:07:21.341366 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 12:07:21.341733 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":421,"threads_started":1}
I20260809 12:07:21.344306 23727 heartbeater.cc:347] Connected to a master server at 127.0.147.126:43625
I20260809 12:07:21.344619 23727 heartbeater.cc:464] Registering TS with master...
I20260809 12:07:21.345346 23727 heartbeater.cc:511] Master 127.0.147.126:43625 requested a full tablet report, sending...
I20260809 12:07:21.346724 23038 ts_manager.cc:194] Registered new tserver with Master: cdf243980d7d40a7bbae1d7d376a0566 (127.0.147.67:45519)
I20260809 12:07:21.347460 23038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:42155
I20260809 12:07:21.353497 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:07:21.371421 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.358082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.0.147.126:43625)
0809 12:07:21.358279 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.358287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.358763 (+ 476us) server_negotiation.cc:407] Connection header received
0809 12:07:21.358973 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.358978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.359049 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.359150 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:21.360692 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.361431 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.362991 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.363397 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.363565 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.366022 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:21.366027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:21.366030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:21.366060 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:21.368689 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:21.369371 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:21.369384 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:21.369386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:21.369525 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:21.369906 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:21.369912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:21.369916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:21.370237 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:21.370502 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.370849 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.371073 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:07:21.376585 23038 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:07:21.385818 23038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:07:21.424381 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.397218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45519 (local address 127.0.0.1:57996)
0809 12:07:21.410399 (+ 13181us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:21.410463 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:21.410691 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:21.413670 (+ 2979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:21.413681 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:21.413725 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:21.414254 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.414271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.418565 (+ 4294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.418574 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.421261 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.421279 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.423296 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.423306 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.423494 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.423618 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 12:07:21.423777 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 12:07:21.423993 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12832,"thread_start_us":12691,"threads_started":1}
I20260809 12:07:21.424371 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.397483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57996 (local address 127.0.147.67:45519)
0809 12:07:21.412850 (+ 15367us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.412864 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.412919 (+ 55us) server_negotiation.cc:407] Connection header received
0809 12:07:21.413044 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.413055 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.413382 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.413623 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:21.415022 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.418336 (+ 3314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.421440 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.422947 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.423094 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.423332 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.423835 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.424002 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15110,"thread_start_us":15165,"threads_started":1}
I20260809 12:07:21.427541 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.397650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47564 (local address 127.0.147.66:35553)
0809 12:07:21.419461 (+ 21811us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.419472 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.419504 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:07:21.419648 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.419655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.419855 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.420811 (+ 956us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:21.421086 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.422558 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.425469 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.426423 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.426591 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.426779 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.427012 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.427167 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21623,"thread_start_us":21478,"threads_started":1}
I20260809 12:07:21.428386 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.397388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:35553 (local address 127.0.0.1:47564)
0809 12:07:21.412378 (+ 14990us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:21.412400 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:21.412545 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:21.420113 (+ 7568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:21.420117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:21.420137 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:21.420596 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.420608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.422707 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.422715 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.425291 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.425313 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.426640 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.426646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.426815 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.426851 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:07:21.426961 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 12:07:21.428085 (+ 1124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14834,"mutex_wait_us":3,"thread_start_us":22097,"threads_started":1}
I20260809 12:07:21.428601 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.411402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42904 (local address 127.0.147.65:45469)
0809 12:07:21.419115 (+ 7713us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.419126 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.420266 (+ 1140us) server_negotiation.cc:407] Connection header received
0809 12:07:21.420375 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.420485 (+ 110us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.420699 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.421594 (+ 895us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:21.421759 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.422824 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.425685 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.427141 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.427279 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.428047 (+ 768us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.428121 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.428276 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7520,"thread_start_us":7377,"threads_started":1}
I20260809 12:07:21.429219 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.411101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:45469 (local address 127.0.0.1:42904)
0809 12:07:21.419797 (+ 8696us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:21.419815 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:21.419944 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:21.420928 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:21.420933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:21.420958 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:21.421394 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.421416 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.422996 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.423001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.425459 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.425477 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.427692 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.427699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.427862 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.427899 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 12:07:21.427989 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 12:07:21.428857 (+ 868us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8567,"mutex_wait_us":704,"thread_start_us":924,"threads_started":1}
I20260809 12:07:21.428839 23621 tablet_service.cc:1467] Processing CreateTablet for tablet daf052fa95834eb0a734423665ade52b (DEFAULT_TABLE table=test-table [id=46fc6b2a1ced425fbd4dff5ee42e2871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:07:21.430516 23621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daf052fa95834eb0a734423665ade52b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:07:21.431720 23427 tablet_service.cc:1467] Processing CreateTablet for tablet daf052fa95834eb0a734423665ade52b (DEFAULT_TABLE table=test-table [id=46fc6b2a1ced425fbd4dff5ee42e2871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:07:21.432827 23427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daf052fa95834eb0a734423665ade52b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:07:21.433043 23217 tablet_service.cc:1467] Processing CreateTablet for tablet daf052fa95834eb0a734423665ade52b (DEFAULT_TABLE table=test-table [id=46fc6b2a1ced425fbd4dff5ee42e2871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:07:21.434357 23217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daf052fa95834eb0a734423665ade52b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:07:21.444136 23752 tablet_bootstrap.cc:492] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: Bootstrap starting.
I20260809 12:07:21.447851 23752 tablet_bootstrap.cc:654] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: Neither blocks nor log segments found. Creating new log.
I20260809 12:07:21.448314 23754 tablet_bootstrap.cc:492] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: Bootstrap starting.
I20260809 12:07:21.449352 23752 log.cc:824] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: Log is configured to *not* fsync() on all Append() calls
I20260809 12:07:21.450982 23755 tablet_bootstrap.cc:492] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: Bootstrap starting.
I20260809 12:07:21.452383 23754 tablet_bootstrap.cc:654] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: Neither blocks nor log segments found. Creating new log.
I20260809 12:07:21.452879 23752 tablet_bootstrap.cc:492] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: No bootstrap required, opened a new log
I20260809 12:07:21.453203 23752 ts_tablet_manager.cc:1397] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 12:07:21.453758 23754 log.cc:824] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: Log is configured to *not* fsync() on all Append() calls
I20260809 12:07:21.454694 23755 tablet_bootstrap.cc:654] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: Neither blocks nor log segments found. Creating new log.
I20260809 12:07:21.455865 23755 log.cc:824] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: Log is configured to *not* fsync() on all Append() calls
I20260809 12:07:21.457320 23754 tablet_bootstrap.cc:492] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: No bootstrap required, opened a new log
I20260809 12:07:21.457633 23754 ts_tablet_manager.cc:1397] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 12:07:21.461670 23752 raft_consensus.cc:348] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:21.462209 23752 raft_consensus.cc:374] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:07:21.462313 23752 raft_consensus.cc:729] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 030d35048f184ebbacc138e11b415cf8, State: Initialized, Role: FOLLOWER
I20260809 12:07:21.463201 23752 consensus_queue.cc:260] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:21.465145 23755 tablet_bootstrap.cc:492] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: No bootstrap required, opened a new log
I20260809 12:07:21.465418 23755 ts_tablet_manager.cc:1397] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20260809 12:07:21.468254 23524 heartbeater.cc:503] Master 127.0.147.126:43625 was elected leader, sending a full tablet report...
I20260809 12:07:21.469044 23752 ts_tablet_manager.cc:1428] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260809 12:07:21.469467 23754 raft_consensus.cc:348] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:21.469906 23754 raft_consensus.cc:374] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:07:21.470021 23754 raft_consensus.cc:729] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdf243980d7d40a7bbae1d7d376a0566, State: Initialized, Role: FOLLOWER
I20260809 12:07:21.470633 23754 consensus_queue.cc:260] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:21.472402 23754 ts_tablet_manager.cc:1428] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: Time spent starting tablet: real 0.015s user 0.007s sys 0.009s
I20260809 12:07:21.472441 23727 heartbeater.cc:503] Master 127.0.147.126:43625 was elected leader, sending a full tablet report...
I20260809 12:07:21.475294 23755 raft_consensus.cc:348] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:21.475739 23755 raft_consensus.cc:374] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:07:21.475862 23755 raft_consensus.cc:729] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cb995fc2b6456dbe4a7af289055d33, State: Initialized, Role: FOLLOWER
I20260809 12:07:21.476408 23755 consensus_queue.cc:260] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:21.477990 23755 ts_tablet_manager.cc:1428] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
W20260809 12:07:21.485287 23320 tablet.cc:2369] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:07:21.518342 23525 tablet.cc:2369] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:07:21.571827 23728 tablet.cc:2369] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:07:21.647193 23766 raft_consensus.cc:482] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:07:21.647471 23766 raft_consensus.cc:504] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:21.649127 23766 leader_election.cc:288] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 030d35048f184ebbacc138e11b415cf8 (127.0.147.66:35553), f7cb995fc2b6456dbe4a7af289055d33 (127.0.147.65:45469)
I20260809 12:07:21.656041 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.649768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:35553 (local address 127.0.147.67:45847)
0809 12:07:21.649951 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:21.649972 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:21.650130 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:21.650607 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:21.650613 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:21.650656 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:21.650850 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.650858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.652123 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.652130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.654035 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.654050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.655393 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.655397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.655575 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.655618 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 12:07:21.655693 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 12:07:21.655822 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 12:07:21.656152 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.649911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:45847 (local address 127.0.147.66:35553)
0809 12:07:21.650105 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.650109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.650127 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:07:21.650310 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.650319 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.650414 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.650561 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:21.650998 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.651981 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.654196 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.655242 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.655403 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.655474 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.655750 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.655889 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:07:21.657148 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.649761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:36955 (local address 127.0.147.65:45469)
0809 12:07:21.649982 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.649988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.650954 (+ 966us) server_negotiation.cc:407] Connection header received
0809 12:07:21.651203 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.651209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.651274 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.651372 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:21.652057 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.652815 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.654853 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.656145 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.656309 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.656380 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.656800 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.656916 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 12:07:21.657266 23768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.649764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:45469 (local address 127.0.147.67:36955)
0809 12:07:21.650845 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:21.650858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:21.651062 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:21.651429 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:21.651433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:21.651456 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:21.651875 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.651894 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.652975 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.652980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.654709 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:21.654720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.656491 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.656497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:21.656626 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.656647 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 12:07:21.656764 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 12:07:21.656882 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":188,"thread_start_us":427,"threads_started":1}
I20260809 12:07:21.657101 23455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daf052fa95834eb0a734423665ade52b" candidate_uuid: "cdf243980d7d40a7bbae1d7d376a0566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "030d35048f184ebbacc138e11b415cf8" is_pre_election: true
I20260809 12:07:21.657662 23455 raft_consensus.cc:2454] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdf243980d7d40a7bbae1d7d376a0566 in term 0.
I20260809 12:07:21.658263 23249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daf052fa95834eb0a734423665ade52b" candidate_uuid: "cdf243980d7d40a7bbae1d7d376a0566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cb995fc2b6456dbe4a7af289055d33" is_pre_election: true
I20260809 12:07:21.658390 23557 leader_election.cc:302] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 030d35048f184ebbacc138e11b415cf8, cdf243980d7d40a7bbae1d7d376a0566; no voters:
I20260809 12:07:21.658773 23766 raft_consensus.cc:2781] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:07:21.658772 23249 raft_consensus.cc:2454] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdf243980d7d40a7bbae1d7d376a0566 in term 0.
I20260809 12:07:21.658937 23766 raft_consensus.cc:482] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:07:21.659045 23766 raft_consensus.cc:3037] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:07:21.661525 23766 raft_consensus.cc:504] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:21.662086 23766 leader_election.cc:288] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [CANDIDATE]: Term 1 election: Requested vote from peers 030d35048f184ebbacc138e11b415cf8 (127.0.147.66:35553), f7cb995fc2b6456dbe4a7af289055d33 (127.0.147.65:45469)
I20260809 12:07:21.662547 23249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daf052fa95834eb0a734423665ade52b" candidate_uuid: "cdf243980d7d40a7bbae1d7d376a0566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cb995fc2b6456dbe4a7af289055d33"
I20260809 12:07:21.662530 23455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daf052fa95834eb0a734423665ade52b" candidate_uuid: "cdf243980d7d40a7bbae1d7d376a0566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "030d35048f184ebbacc138e11b415cf8"
I20260809 12:07:21.662744 23249 raft_consensus.cc:3037] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:07:21.662756 23455 raft_consensus.cc:3037] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:07:21.666042 23249 raft_consensus.cc:2454] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdf243980d7d40a7bbae1d7d376a0566 in term 1.
I20260809 12:07:21.666033 23455 raft_consensus.cc:2454] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdf243980d7d40a7bbae1d7d376a0566 in term 1.
I20260809 12:07:21.666441 23554 leader_election.cc:302] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdf243980d7d40a7bbae1d7d376a0566, f7cb995fc2b6456dbe4a7af289055d33; no voters:
I20260809 12:07:21.666728 23766 raft_consensus.cc:2781] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:07:21.667152 23766 raft_consensus.cc:686] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 1 LEADER]: Becoming Leader. State: Replica: cdf243980d7d40a7bbae1d7d376a0566, State: Running, Role: LEADER
I20260809 12:07:21.667662 23766 consensus_queue.cc:237] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:21.672052 23036 catalog_manager.cc:5495] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdf243980d7d40a7bbae1d7d376a0566 (127.0.147.67). New cstate: current_term: 1 leader_uuid: "cdf243980d7d40a7bbae1d7d376a0566" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } health_report { overall_health: UNKNOWN } } }
I20260809 12:07:21.701787 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.696732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.0.147.67:45519)
0809 12:07:21.696910 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.696915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.697016 (+ 101us) server_negotiation.cc:407] Connection header received
0809 12:07:21.697177 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.697181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.697258 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.697383 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:07:21.697693 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.698401 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.699929 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.700310 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.700401 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.700835 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:07:21.701334 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:07:21.701446 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.701517 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.701587 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 12:07:21.709630 23249 raft_consensus.cc:1261] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [term 1 FOLLOWER]: Refusing update from remote peer cdf243980d7d40a7bbae1d7d376a0566: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:07:21.709647 23455 raft_consensus.cc:1261] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [term 1 FOLLOWER]: Refusing update from remote peer cdf243980d7d40a7bbae1d7d376a0566: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:07:21.710564 23769 consensus_queue.cc:1035] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [LEADER]: Connected to new peer: Peer: permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:07:21.710938 23766 consensus_queue.cc:1035] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:07:21.736820 23036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:07:21.737178 23036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:07:21.740834 23036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Sending DeleteTablet for 3 replicas of tablet daf052fa95834eb0a734423665ade52b
I20260809 12:07:21.741925 23621 tablet_service.cc:1514] Processing DeleteTablet for tablet daf052fa95834eb0a734423665ade52b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:07:21 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:57996
I20260809 12:07:21.742121 23427 tablet_service.cc:1514] Processing DeleteTablet for tablet daf052fa95834eb0a734423665ade52b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:07:21 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:47564
I20260809 12:07:21.742966 23790 tablet_replica.cc:330] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: stopping tablet replica
I20260809 12:07:21.743058 23217 tablet_service.cc:1514] Processing DeleteTablet for tablet daf052fa95834eb0a734423665ade52b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:07:21 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:42904
I20260809 12:07:21.743641 23790 raft_consensus.cc:2229] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:07:21.744639 23790 raft_consensus.cc:2258] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:07:21.745015 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 23126
I20260809 12:07:21.746025 23791 tablet_replica.cc:330] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: stopping tablet replica
I20260809 12:07:21.746449 23791 raft_consensus.cc:2229] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:07:21.747462 23791 raft_consensus.cc:2258] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:07:21.748536 23790 ts_tablet_manager.cc:1905] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:07:21.750159 23791 ts_tablet_manager.cc:1905] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:07:21.757406 23790 ts_tablet_manager.cc:1918] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:07:21.757707 23790 log.cc:1196] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/daf052fa95834eb0a734423665ade52b
I20260809 12:07:21.758374 23790 ts_tablet_manager.cc:1939] T daf052fa95834eb0a734423665ade52b P cdf243980d7d40a7bbae1d7d376a0566: Deleting consensus metadata
I20260809 12:07:21.759090 23791 ts_tablet_manager.cc:1918] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:07:21.759275 23791 log.cc:1196] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/daf052fa95834eb0a734423665ade52b
I20260809 12:07:21.759846 23791 ts_tablet_manager.cc:1939] T daf052fa95834eb0a734423665ade52b P 030d35048f184ebbacc138e11b415cf8: Deleting consensus metadata
I20260809 12:07:21.760926 23792 tablet_replica.cc:330] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: stopping tablet replica
I20260809 12:07:21.760865 23015 catalog_manager.cc:4841] TS cdf243980d7d40a7bbae1d7d376a0566 (127.0.147.67:45519): tablet daf052fa95834eb0a734423665ade52b (table test-table [id=46fc6b2a1ced425fbd4dff5ee42e2871]) successfully deleted
I20260809 12:07:21.761361 23792 raft_consensus.cc:2229] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:07:21.761830 23792 raft_consensus.cc:2258] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:07:21.762385 23017 catalog_manager.cc:4841] TS 030d35048f184ebbacc138e11b415cf8 (127.0.147.66:35553): tablet daf052fa95834eb0a734423665ade52b (table test-table [id=46fc6b2a1ced425fbd4dff5ee42e2871]) successfully deleted
I20260809 12:07:21.763839 23792 ts_tablet_manager.cc:1905] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 12:07:21 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:07:21.773088 23792 ts_tablet_manager.cc:1918] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:07:21.773268 23792 log.cc:1196] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/daf052fa95834eb0a734423665ade52b
I20260809 12:07:21.773792 23792 ts_tablet_manager.cc:1939] T daf052fa95834eb0a734423665ade52b P f7cb995fc2b6456dbe4a7af289055d33: Deleting consensus metadata
I20260809 12:07:21.775748 23015 catalog_manager.cc:4841] TS f7cb995fc2b6456dbe4a7af289055d33 (127.0.147.65:45469): tablet daf052fa95834eb0a734423665ade52b (table test-table [id=46fc6b2a1ced425fbd4dff5ee42e2871]) successfully deleted
I20260809 12:07:21.777055 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.747181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42912 (local address 127.0.147.65:45469)
0809 12:07:21.747351 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.747358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.756879 (+ 9521us) server_negotiation.cc:407] Connection header received
0809 12:07:21.757396 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.757405 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.757498 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.757608 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:21.759967 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.761191 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.763232 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.763842 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.763983 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.768603 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:21.768617 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:21.768633 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:21.768683 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:21.772923 (+ 4240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:21.774116 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:21.774125 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:21.774128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:21.774295 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:21.774783 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:21.774787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:21.774790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:21.775268 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:21.776230 (+ 962us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.776410 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.776792 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:07:21.777719 23310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:21.869813 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 23126
I20260809 12:07:21.902063 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 23327
Aug 09 12:07:21 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:07:21.918267 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:21.902951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47572 (local address 127.0.147.66:35553)
0809 12:07:21.903190 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:21.903196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:21.903217 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:07:21.903355 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:21.903359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:21.903423 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:21.903532 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:21.904906 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.905690 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.907020 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:21.907408 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:21.907572 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:21.912322 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:21.912338 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:21.912353 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:21.912441 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:21.915306 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:21.916069 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:21.916085 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:21.916087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:21.916221 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:21.916650 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:21.916653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:21.916655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:21.917013 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:21.917332 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:21.917707 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 12:07:21.917932 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 12:07:21.918973 23517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:22.011068 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 23327
I20260809 12:07:22.043467 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 23531
Aug 09 12:07:22 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:07:22.062340 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:22.044312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.0.147.67:45519)
0809 12:07:22.044539 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:22.044548 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:22.044629 (+ 81us) server_negotiation.cc:407] Connection header received
0809 12:07:22.044830 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:22.044835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:22.044910 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:22.045021 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:22.046438 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:22.047277 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:22.048831 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:22.049320 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:22.049466 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:22.054722 (+ 5256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:22.054735 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:22.054754 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:22.054799 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:22.059231 (+ 4432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:22.060030 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:22.060041 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:22.060044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:22.060237 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:22.060619 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:22.060624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:22.060628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:22.061100 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:22.061460 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:22.061683 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 12:07:22.061933 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 12:07:22.063205 23719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:22.186617 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 23531
I20260809 12:07:22.215730 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 22991
I20260809 12:07:22.216930 23083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:22.329203 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 22991
I20260809 12:07:42.357457 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:43625
--webserver_interface=127.0.147.126
--webserver_port=43635
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:43625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:07:42.710028 23803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:42.710317 23803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:42.710409 23803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:42.717518 23803 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 12:07:42.717624 23803 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 12:07:42.717665 23803 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 12:07:42.717711 23803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:07:42.717767 23803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:42.717837 23803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:42.717885 23803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:07:42.717933 23803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:07:42.727778 23803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:43625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:43625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=43635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:42.728997 23803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:42.730233 23803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:42.757301 23811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:42.757400 23812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:42.757333 23815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:42.758271 23803 server_base.cc:1034] running on GCE node
I20260809 12:07:42.759204 23803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:42.760872 23803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:42.762168 23803 hybrid_clock.cc:648] HybridClock initialized: now 1786277262762106 us; error 38 us; skew 500 ppm
I20260809 12:07:42.762537 23803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:42 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277262, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:42.770211 23803 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:07:42.772202 23803 webserver.cc:460] Webserver started at http://127.0.147.126:43635/ using document root <none> and password file <none>
I20260809 12:07:42.773080 23803 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:42.773197 23803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:42.779927 23803 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:07:42.783092 23825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:42.784641 23803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:07:42.784855 23803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8"
format_stamp: "Formatted at 2026-08-09 12:07:19 on dist-test-slave-sb2z"
server_key: "d6e08ed0a8817f31249c40a4d42f0014"
server_key_iv: "e20538a5b23e4b4f036430e125f11291"
server_key_version: "encryptionkey@0"
I20260809 12:07:42.786055 23803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:42.800818 23803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:42.805889 23803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:42.806226 23803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:42.827625 23903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:43625 every 8 connection(s)
I20260809 12:07:42.827688 23803 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:43625
I20260809 12:07:42.829376 23803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:07:42.832787 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 23803
I20260809 12:07:42.833899 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:45469
--local_ip_for_outbound_sockets=127.0.147.65
--tserver_master_addrs=127.0.147.126:43625
--webserver_port=42129
--webserver_interface=127.0.147.65
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:07:42.839023 23905 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:07:42.844413 23905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Bootstrap starting.
I20260809 12:07:42.866163 23905 log.cc:824] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Log is configured to *not* fsync() on all Append() calls
I20260809 12:07:42.878347 23905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:07:42.878901 23905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Bootstrap complete.
I20260809 12:07:42.888296 23905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:07:42.889143 23905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a45a9d2ff94c2c8f84a9151d88a6f8, State: Initialized, Role: FOLLOWER
I20260809 12:07:42.889879 23905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:07:42.890153 23905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:07:42.890276 23905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:07:42.890465 23905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:07:42.894506 23905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:07:42.895030 23905 leader_election.cc:302] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32a45a9d2ff94c2c8f84a9151d88a6f8; no voters:
I20260809 12:07:42.895740 23905 leader_election.cc:288] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:07:42.895887 23910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:07:42.897315 23910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 2 LEADER]: Becoming Leader. State: Replica: 32a45a9d2ff94c2c8f84a9151d88a6f8, State: Running, Role: LEADER
I20260809 12:07:42.897904 23910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:07:42.898124 23905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:07:42.900635 23911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } } }
I20260809 12:07:42.901103 23911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:07:42.905834 23912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32a45a9d2ff94c2c8f84a9151d88a6f8. Latest consensus state: current_term: 2 leader_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } } }
I20260809 12:07:42.906769 23912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:07:42.908645 23917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:07:42.917799 23917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:07:42.919348 23917 catalog_manager.cc:1259] Loaded cluster ID: 5d121e48a55b448dac144fcd384cb1d9
I20260809 12:07:42.919513 23917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:07:42.923821 23917 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:07:42.925369 23917 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Loaded TSK: 0
I20260809 12:07:42.972076 23917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Generated new TSK 1
I20260809 12:07:42.972529 23917 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:07:43.209121 23907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:43.209456 23907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:43.209638 23907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:43.217249 23907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:43.217427 23907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:43.217602 23907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:07:43.227942 23907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:45469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=42129
--tserver_master_addrs=127.0.147.126:43625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:43.229312 23907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:43.230705 23907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:43.245975 23938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:43.246057 23940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:43.246353 23942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:43.247058 23907 server_base.cc:1034] running on GCE node
I20260809 12:07:43.247784 23907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:43.248741 23907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:43.249993 23907 hybrid_clock.cc:648] HybridClock initialized: now 1786277263249932 us; error 48 us; skew 500 ppm
I20260809 12:07:43.250324 23907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:43 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277263, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:43.253713 23907 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:07:43.255352 23907 webserver.cc:460] Webserver started at http://127.0.147.65:42129/ using document root <none> and password file <none>
I20260809 12:07:43.256249 23907 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:43.256428 23907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:43.261992 23907 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 12:07:43.264804 23951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:43.266150 23907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:07:43.266335 23907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "f7cb995fc2b6456dbe4a7af289055d33"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "ccd7b233da22769135738670803fb8b8"
server_key_iv: "522c25a6f39d4807c7dd11f5a38e354d"
server_key_version: "encryptionkey@0"
I20260809 12:07:43.267331 23907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:43.297704 23907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:43.301939 23907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:43.302232 23907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:43.303292 23907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:07:43.304791 23907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:07:43.304912 23907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:43.305028 23907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:07:43.305111 23907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:43.347791 23907 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:45469
I20260809 12:07:43.347898 24115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:45469 every 8 connection(s)
I20260809 12:07:43.349291 23907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:07:43.359294 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 23907
I20260809 12:07:43.360178 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:35553
--local_ip_for_outbound_sockets=127.0.147.66
--tserver_master_addrs=127.0.147.126:43625
--webserver_port=40569
--webserver_interface=127.0.147.66
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:07:43 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277263, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:07:43.374832 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:43.352546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:54823 (local address 127.0.147.126:43625)
0809 12:07:43.353149 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:43.353160 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:43.353737 (+ 577us) server_negotiation.cc:407] Connection header received
0809 12:07:43.355998 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:43.356012 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:43.356351 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:43.356650 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:43.358065 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:43.359286 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:43.361857 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:43.362516 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:43.362657 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:43.366848 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:43.366856 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:43.366869 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:43.366911 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:43.369876 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:43.370790 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:43.370796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:43.370802 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:43.370962 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:43.371577 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:43.371582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:43.371586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:43.372103 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:43.372327 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:43.373182 (+ 855us) server_negotiation.cc:299] Negotiation successful
0809 12:07:43.373578 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":270,"threads_started":1}
I20260809 12:07:43.374825 24121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:43.352731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:54823)
0809 12:07:43.353560 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:43.353613 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:43.355397 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:43.356794 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:43.356801 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:43.357363 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:07:43.357876 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:43.357892 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:43.359540 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:43.359547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:43.361704 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:43.361725 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:43.362761 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:43.362771 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:43.363073 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:43.363884 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:07:43.363917 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:07:43.366568 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:07:43.370038 (+ 3470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:43.370049 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:43.370062 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:43.370614 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:43.371176 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:43.371180 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:43.371182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:43.371356 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:43.372255 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:07:43.372262 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:07:43.372603 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 12:07:43.372920 (+ 317us) client_negotiation.cc:240] Negotiation successful
0809 12:07:43.373479 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":360,"threads_started":1}
I20260809 12:07:43.377815 24117 heartbeater.cc:347] Connected to a master server at 127.0.147.126:43625
I20260809 12:07:43.378155 24117 heartbeater.cc:464] Registering TS with master...
I20260809 12:07:43.379071 24117 heartbeater.cc:511] Master 127.0.147.126:43625 requested a full tablet report, sending...
I20260809 12:07:43.381373 23852 ts_manager.cc:194] Registered new tserver with Master: f7cb995fc2b6456dbe4a7af289055d33 (127.0.147.65:45469)
I20260809 12:07:43.382766 23852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:54823
W20260809 12:07:43.713038 24122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:43.713327 24122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:43.713502 24122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:43.720599 24122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:43.720700 24122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:43.720929 24122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:07:43.730290 24122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:35553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=40569
--tserver_master_addrs=127.0.147.126:43625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:43.731580 24122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:43.732893 24122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:43.746235 24128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:43.746232 24127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:43.746584 24131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:43.747185 24122 server_base.cc:1034] running on GCE node
I20260809 12:07:43.747876 24122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:43.748821 24122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:43.750025 24122 hybrid_clock.cc:648] HybridClock initialized: now 1786277263749980 us; error 33 us; skew 500 ppm
I20260809 12:07:43.750344 24122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:43 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277263, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:43.753611 24122 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:07:43.755363 24122 webserver.cc:460] Webserver started at http://127.0.147.66:40569/ using document root <none> and password file <none>
I20260809 12:07:43.756282 24122 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:43.756428 24122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:43.762184 24122 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 12:07:43.764966 24143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:43.766321 24122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:07:43.766513 24122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "030d35048f184ebbacc138e11b415cf8"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "c209bfc9272956a244ebf9ff3b778685"
server_key_iv: "8740f224a76ffbeecb3a3c7fdba55ea3"
server_key_version: "encryptionkey@0"
I20260809 12:07:43.767522 24122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:43.798038 24122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:43.802472 24122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:43.802762 24122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:43.803817 24122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:07:43.805166 24122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:07:43.805338 24122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:43.805473 24122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:07:43.805568 24122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:43.843377 24122 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:35553
I20260809 12:07:43.843436 24311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:35553 every 8 connection(s)
I20260809 12:07:43.844807 24122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:07:43.845523 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 24122
I20260809 12:07:43.846187 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:45519
--local_ip_for_outbound_sockets=127.0.147.67
--tserver_master_addrs=127.0.147.126:43625
--webserver_port=40529
--webserver_interface=127.0.147.67
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:07:43 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277263, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:07:43.873318 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:43.849520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:50719 (local address 127.0.147.126:43625)
0809 12:07:43.849709 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:43.849716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:43.850006 (+ 290us) server_negotiation.cc:407] Connection header received
0809 12:07:43.852153 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:43.852159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:43.852266 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:43.852387 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:43.853978 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:43.854684 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:43.861629 (+ 6945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:43.862430 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:43.862590 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:43.866726 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:43.866731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:43.866735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:43.866769 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:43.869498 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:43.870583 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:43.870587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:43.870589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:43.870716 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:43.871311 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:43.871317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:43.871324 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:43.871701 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:43.871826 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:43.872833 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0809 12:07:43.873050 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:07:43.874946 24316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:43.848831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:50719)
0809 12:07:43.849823 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:43.849870 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:43.851867 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:43.852639 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:43.852647 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:43.853183 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:07:43.853755 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:43.853772 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:43.854855 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:43.854860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:43.861408 (+ 6548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:43.861449 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:43.862669 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:43.862674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:43.862877 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:43.863692 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:07:43.863717 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:07:43.866439 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:07:43.869659 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:43.869670 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:43.869690 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:43.870420 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:43.870838 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:43.870846 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:43.870852 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:43.871172 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:43.871836 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:07:43.871847 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:07:43.872398 (+ 551us) client_negotiation.cc:769] Sending connection context
0809 12:07:43.872724 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 12:07:43.873335 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":407,"threads_started":1}
I20260809 12:07:43.876696 24313 heartbeater.cc:347] Connected to a master server at 127.0.147.126:43625
I20260809 12:07:43.877020 24313 heartbeater.cc:464] Registering TS with master...
I20260809 12:07:43.877750 24313 heartbeater.cc:511] Master 127.0.147.126:43625 requested a full tablet report, sending...
I20260809 12:07:43.879208 23852 ts_manager.cc:194] Registered new tserver with Master: 030d35048f184ebbacc138e11b415cf8 (127.0.147.66:35553)
I20260809 12:07:43.880110 23852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:50719
W20260809 12:07:44.209183 24317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:07:44.209542 24317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:07:44.209703 24317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:07:44.217142 24317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:07:44.217285 24317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:07:44.217504 24317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:07:44.227867 24317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:45519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=40529
--tserver_master_addrs=127.0.147.126:43625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:07:44.229199 24317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:07:44.230501 24317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:07:44.244421 24324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:44.244509 24327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:07:44.244447 24323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:07:44.245394 24317 server_base.cc:1034] running on GCE node
I20260809 12:07:44.246026 24317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:07:44.246940 24317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:07:44.248180 24317 hybrid_clock.cc:648] HybridClock initialized: now 1786277264248129 us; error 58 us; skew 500 ppm
I20260809 12:07:44.248502 24317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:07:44 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277264, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:07:44.251742 24317 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:07:44.253203 24317 webserver.cc:460] Webserver started at http://127.0.147.67:40529/ using document root <none> and password file <none>
I20260809 12:07:44.253865 24317 fs_manager.cc:359] Metadata directory not provided
I20260809 12:07:44.254024 24317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:07:44.259388 24317 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:07:44.261982 24337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:44.263352 24317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:07:44.263589 24317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "cdf243980d7d40a7bbae1d7d376a0566"
format_stamp: "Formatted at 2026-08-09 12:07:21 on dist-test-slave-sb2z"
server_key: "7f6f6ceca4a44db2348ec09830262d99"
server_key_iv: "83dfd9a6d851378ea8a0d182e724efb9"
server_key_version: "encryptionkey@0"
I20260809 12:07:44.264668 24317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:07:44.279266 24317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:07:44.283725 24317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:07:44.283990 24317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:07:44.284896 24317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:07:44.286178 24317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:07:44.286302 24317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:44.286418 24317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:07:44.286511 24317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:07:44.324620 24317 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:45519
I20260809 12:07:44.324709 24504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:45519 every 8 connection(s)
I20260809 12:07:44.326130 24317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:07:44.334095 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 24317
Aug 09 12:07:44 dist-test-slave-sb2z krb5kdc[22975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277264, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:07:44.348730 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.328730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:42631 (local address 127.0.147.126:43625)
0809 12:07:44.328952 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.328958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.330130 (+ 1172us) server_negotiation.cc:407] Connection header received
0809 12:07:44.332040 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.332044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.332110 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.332201 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:44.333727 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.334469 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.336531 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.337228 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.337384 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.341467 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:44.341473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:44.341478 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:44.341552 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:44.345136 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.345848 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.345852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.345854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.345980 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.346504 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.346514 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.346522 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.346898 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:44.347050 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.348006 (+ 956us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.348280 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:07:44.349841 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.329203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:42631)
0809 12:07:44.329977 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:44.330016 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:44.331780 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:44.332403 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:44.332413 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:44.332968 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:07:44.333560 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.333579 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.334645 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.334651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.336384 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.336396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.337467 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.337472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.337701 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.338473 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:07:44.338508 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:07:44.341258 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:07:44.345290 (+ 4032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:44.345296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:44.345307 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:44.345707 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:44.346181 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:07:44.346190 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:07:44.346193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:07:44.346380 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:07:44.347064 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:07:44.347075 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:07:44.347509 (+ 434us) client_negotiation.cc:769] Sending connection context
0809 12:07:44.347864 (+ 355us) client_negotiation.cc:240] Negotiation successful
0809 12:07:44.348447 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":310,"threads_started":1}
I20260809 12:07:44.351363 24505 heartbeater.cc:347] Connected to a master server at 127.0.147.126:43625
I20260809 12:07:44.351718 24505 heartbeater.cc:464] Registering TS with master...
I20260809 12:07:44.352524 24505 heartbeater.cc:511] Master 127.0.147.126:43625 requested a full tablet report, sending...
I20260809 12:07:44.352841 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.335074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44350 (local address 127.0.147.126:43625)
0809 12:07:44.336007 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.336013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.336026 (+ 13us) server_negotiation.cc:407] Connection header received
0809 12:07:44.336063 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.336066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.336116 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.336227 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:44.338359 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.339516 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.341497 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.341994 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.342089 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.345640 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:44.345644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:44.345647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:44.345674 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:44.349147 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.350051 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.350056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.350059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.350206 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.350980 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.350985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.350989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.351362 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:44.351639 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.352198 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.352427 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":648,"threads_started":1}
I20260809 12:07:44.354326 23852 ts_manager.cc:194] Registered new tserver with Master: cdf243980d7d40a7bbae1d7d376a0566 (127.0.147.67:45519)
I20260809 12:07:44.355274 23852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:42631
I20260809 12:07:44.356164 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:07:44.362989 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.357092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44358 (local address 127.0.147.126:43625)
0809 12:07:44.357250 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.357254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.357736 (+ 482us) server_negotiation.cc:407] Connection header received
0809 12:07:44.357976 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.357980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.358095 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.358235 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:07:44.358649 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.359319 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.360871 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.361321 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.361452 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.362042 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:07:44.362293 (+ 251us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 12:07:44.362673 (+ 380us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44358: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 12:07:44.363123 24510 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44358: token expired
W20260809 12:07:44.363700 23737 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.147.126:43625)
I20260809 12:07:44.377282 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.364238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44360 (local address 127.0.147.126:43625)
0809 12:07:44.364407 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.364412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.364430 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:07:44.364563 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.364568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.364630 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.364725 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:44.366113 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.366807 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.368432 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.369331 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.369472 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.372295 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:44.372300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:44.372303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:44.372332 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:44.374658 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.375404 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.375414 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.375417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.375588 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.376022 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.376026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.376028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.376318 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:44.376512 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.376822 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.377034 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:07:44.380654 23851 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 12:07:44.386672 24117 heartbeater.cc:503] Master 127.0.147.126:43625 was elected leader, sending a full tablet report...
W20260809 12:07:44.388351 23851 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:07:44.421669 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.400258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51702 (local address 127.0.147.67:45519)
0809 12:07:44.413673 (+ 13415us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.413682 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.413716 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:07:44.413817 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.413824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.414019 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.414986 (+ 967us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:44.415215 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.416593 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.418975 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.420331 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.420473 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.420728 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.421088 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.421264 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13226,"thread_start_us":13068,"threads_started":1}
I20260809 12:07:44.421698 24518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.399817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:45519 (local address 127.0.0.1:51702)
0809 12:07:44.400974 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:44.401018 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:44.401189 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:44.414255 (+ 13066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:44.414260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:44.414292 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:44.414791 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.414805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.416746 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.416752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.418810 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.418824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.420585 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.420592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.420816 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.420870 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 12:07:44.421024 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 12:07:44.421216 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":29,"thread_start_us":762,"threads_started":1}
I20260809 12:07:44.422652 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.401368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:45469 (local address 127.0.0.1:41644)
0809 12:07:44.412692 (+ 11324us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:44.412710 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:44.413057 (+ 347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:44.414023 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:44.414030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:44.414083 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:44.414633 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.414650 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.416685 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.416693 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.419267 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.419582 (+ 315us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.421758 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.421766 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.421970 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.422012 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 12:07:44.422115 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 12:07:44.422247 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11201,"mutex_wait_us":68,"thread_start_us":11478,"threads_started":1}
I20260809 12:07:44.422705 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.401849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.0.147.65:45469)
0809 12:07:44.412151 (+ 10302us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.412165 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.412819 (+ 654us) server_negotiation.cc:407] Connection header received
0809 12:07:44.413289 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.413302 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.413565 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.413769 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:44.415069 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.416532 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.419741 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.421225 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.421362 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.421578 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.422199 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.422362 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10045,"thread_start_us":10284,"threads_started":1}
I20260809 12:07:44.423242 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.400811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.0.147.66:35553)
0809 12:07:44.412910 (+ 12099us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.412919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.412945 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:07:44.413046 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.413053 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.413368 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.413584 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:44.416174 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.417739 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.420133 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.421528 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.421672 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.421885 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.422754 (+ 869us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.422920 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11921,"thread_start_us":11816,"threads_started":1}
I20260809 12:07:44.423262 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.400148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:35553 (local address 127.0.0.1:58910)
0809 12:07:44.410511 (+ 10363us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:44.410535 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:44.410708 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:44.413641 (+ 2933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:44.415605 (+ 1964us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:44.415647 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:44.415984 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.415995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.417891 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.417897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.419966 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.419980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.422053 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.422059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.422242 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.422281 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:07:44.422379 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 12:07:44.422527 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10185,"thread_start_us":10548,"threads_started":1}
I20260809 12:07:44.426397 24021 tablet_service.cc:1467] Processing CreateTablet for tablet 7efee9b586b54e23af11d89340928118 (DEFAULT_TABLE table=test-table [id=db34fd5d29344beb926699d7aa9b2d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:07:44.426478 24406 tablet_service.cc:1467] Processing CreateTablet for tablet 7efee9b586b54e23af11d89340928118 (DEFAULT_TABLE table=test-table [id=db34fd5d29344beb926699d7aa9b2d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:07:44.426612 24217 tablet_service.cc:1467] Processing CreateTablet for tablet 7efee9b586b54e23af11d89340928118 (DEFAULT_TABLE table=test-table [id=db34fd5d29344beb926699d7aa9b2d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:07:44.427850 24021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efee9b586b54e23af11d89340928118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:07:44.428059 24406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efee9b586b54e23af11d89340928118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:07:44.428067 24217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efee9b586b54e23af11d89340928118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:07:44.441476 24527 tablet_bootstrap.cc:492] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: Bootstrap starting.
I20260809 12:07:44.444181 24528 tablet_bootstrap.cc:492] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: Bootstrap starting.
I20260809 12:07:44.445540 24527 tablet_bootstrap.cc:654] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: Neither blocks nor log segments found. Creating new log.
I20260809 12:07:44.446334 24529 tablet_bootstrap.cc:492] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: Bootstrap starting.
I20260809 12:07:44.447125 24527 log.cc:824] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: Log is configured to *not* fsync() on all Append() calls
I20260809 12:07:44.447988 24528 tablet_bootstrap.cc:654] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: Neither blocks nor log segments found. Creating new log.
I20260809 12:07:44.449079 24528 log.cc:824] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: Log is configured to *not* fsync() on all Append() calls
I20260809 12:07:44.450171 24529 tablet_bootstrap.cc:654] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: Neither blocks nor log segments found. Creating new log.
I20260809 12:07:44.450687 24527 tablet_bootstrap.cc:492] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: No bootstrap required, opened a new log
I20260809 12:07:44.450994 24527 ts_tablet_manager.cc:1397] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260809 12:07:44.451431 24529 log.cc:824] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: Log is configured to *not* fsync() on all Append() calls
I20260809 12:07:44.452339 24528 tablet_bootstrap.cc:492] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: No bootstrap required, opened a new log
I20260809 12:07:44.452601 24528 ts_tablet_manager.cc:1397] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 12:07:44.456009 24529 tablet_bootstrap.cc:492] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: No bootstrap required, opened a new log
I20260809 12:07:44.456277 24529 ts_tablet_manager.cc:1397] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260809 12:07:44.462672 24527 raft_consensus.cc:348] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:44.463195 24527 raft_consensus.cc:374] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:07:44.463303 24527 raft_consensus.cc:729] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cb995fc2b6456dbe4a7af289055d33, State: Initialized, Role: FOLLOWER
I20260809 12:07:44.463393 24528 raft_consensus.cc:348] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:44.463877 24528 raft_consensus.cc:374] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:07:44.464004 24528 raft_consensus.cc:729] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdf243980d7d40a7bbae1d7d376a0566, State: Initialized, Role: FOLLOWER
I20260809 12:07:44.464054 24527 consensus_queue.cc:260] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:44.464162 24529 raft_consensus.cc:348] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:44.464674 24529 raft_consensus.cc:374] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:07:44.464627 24528 consensus_queue.cc:260] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:44.464808 24529 raft_consensus.cc:729] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 030d35048f184ebbacc138e11b415cf8, State: Initialized, Role: FOLLOWER
I20260809 12:07:44.465432 24529 consensus_queue.cc:260] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:44.468199 24527 ts_tablet_manager.cc:1428] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: Time spent starting tablet: real 0.017s user 0.016s sys 0.001s
I20260809 12:07:44.470016 24313 heartbeater.cc:503] Master 127.0.147.126:43625 was elected leader, sending a full tablet report...
I20260809 12:07:44.470304 24528 ts_tablet_manager.cc:1428] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260809 12:07:44.470631 24505 heartbeater.cc:503] Master 127.0.147.126:43625 was elected leader, sending a full tablet report...
I20260809 12:07:44.470903 24529 ts_tablet_manager.cc:1428] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: Time spent starting tablet: real 0.014s user 0.007s sys 0.008s
I20260809 12:07:44.529202 24535 raft_consensus.cc:482] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:07:44.529522 24535 raft_consensus.cc:504] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:44.531239 24535 leader_election.cc:288] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 030d35048f184ebbacc138e11b415cf8 (127.0.147.66:35553), f7cb995fc2b6456dbe4a7af289055d33 (127.0.147.65:45469)
I20260809 12:07:44.538599 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.531693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:45469 (local address 127.0.147.67:39627)
0809 12:07:44.531945 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:44.531969 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:44.532094 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:44.532509 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:44.532516 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:44.532555 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:44.532800 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.532815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.534285 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.534291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.536442 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.536458 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.537922 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.537926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.538085 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.538111 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 12:07:44.538191 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 12:07:44.538284 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 12:07:44.538735 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.531911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:39627 (local address 127.0.147.65:45469)
0809 12:07:44.532116 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.532122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.532143 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:07:44.532223 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.532231 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.532325 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.532455 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:44.532987 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.533996 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.536632 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.537767 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.537944 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.538024 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.538276 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.538452 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 12:07:44.539345 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.531809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:45557 (local address 127.0.147.66:35553)
0809 12:07:44.532033 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.532039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.533010 (+ 971us) server_negotiation.cc:407] Connection header received
0809 12:07:44.533154 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.533160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.533235 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.533337 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:07:44.533832 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.534619 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.536930 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.538301 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.538460 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.538526 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.538985 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.539099 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:07:44.540062 24538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.532126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:35553 (local address 127.0.147.67:45557)
0809 12:07:44.532858 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 12:07:44.532878 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:07:44.533036 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:07:44.533349 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:07:44.533352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:07:44.533369 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:07:44.533669 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.533676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.534792 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.534799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.536790 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:07:44.536800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.538555 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.538562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:07:44.538746 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.538805 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 12:07:44.539572 (+ 767us) client_negotiation.cc:240] Negotiation successful
0809 12:07:44.539717 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":350,"threads_started":1}
I20260809 12:07:44.539852 24053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efee9b586b54e23af11d89340928118" candidate_uuid: "cdf243980d7d40a7bbae1d7d376a0566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cb995fc2b6456dbe4a7af289055d33" is_pre_election: true
I20260809 12:07:44.540493 24053 raft_consensus.cc:2454] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdf243980d7d40a7bbae1d7d376a0566 in term 0.
I20260809 12:07:44.541294 24341 leader_election.cc:302] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdf243980d7d40a7bbae1d7d376a0566, f7cb995fc2b6456dbe4a7af289055d33; no voters:
I20260809 12:07:44.541219 24245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efee9b586b54e23af11d89340928118" candidate_uuid: "cdf243980d7d40a7bbae1d7d376a0566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "030d35048f184ebbacc138e11b415cf8" is_pre_election: true
I20260809 12:07:44.541774 24535 raft_consensus.cc:2781] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:07:44.541786 24245 raft_consensus.cc:2454] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdf243980d7d40a7bbae1d7d376a0566 in term 0.
I20260809 12:07:44.541940 24535 raft_consensus.cc:482] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:07:44.542059 24535 raft_consensus.cc:3037] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:07:44.544437 24535 raft_consensus.cc:504] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:44.545137 24535 leader_election.cc:288] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [CANDIDATE]: Term 1 election: Requested vote from peers 030d35048f184ebbacc138e11b415cf8 (127.0.147.66:35553), f7cb995fc2b6456dbe4a7af289055d33 (127.0.147.65:45469)
I20260809 12:07:44.545446 24245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efee9b586b54e23af11d89340928118" candidate_uuid: "cdf243980d7d40a7bbae1d7d376a0566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "030d35048f184ebbacc138e11b415cf8"
I20260809 12:07:44.545650 24245 raft_consensus.cc:3037] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:07:44.545589 24053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efee9b586b54e23af11d89340928118" candidate_uuid: "cdf243980d7d40a7bbae1d7d376a0566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cb995fc2b6456dbe4a7af289055d33"
I20260809 12:07:44.545840 24053 raft_consensus.cc:3037] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:07:44.547999 24245 raft_consensus.cc:2454] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdf243980d7d40a7bbae1d7d376a0566 in term 1.
I20260809 12:07:44.548189 24053 raft_consensus.cc:2454] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdf243980d7d40a7bbae1d7d376a0566 in term 1.
I20260809 12:07:44.548456 24344 leader_election.cc:302] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 030d35048f184ebbacc138e11b415cf8, cdf243980d7d40a7bbae1d7d376a0566; no voters:
I20260809 12:07:44.548771 24535 raft_consensus.cc:2781] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:07:44.549254 24535 raft_consensus.cc:686] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 1 LEADER]: Becoming Leader. State: Replica: cdf243980d7d40a7bbae1d7d376a0566, State: Running, Role: LEADER
I20260809 12:07:44.549723 24535 consensus_queue.cc:237] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } }
I20260809 12:07:44.554323 23852 catalog_manager.cc:5495] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdf243980d7d40a7bbae1d7d376a0566 (127.0.147.67). New cstate: current_term: 1 leader_uuid: "cdf243980d7d40a7bbae1d7d376a0566" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf243980d7d40a7bbae1d7d376a0566" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 45519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 } health_report { overall_health: UNKNOWN } } }
W20260809 12:07:44.577788 24506 tablet.cc:2369] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:07:44.599365 24314 tablet.cc:2369] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:07:44.601820 24118 tablet.cc:2369] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:07:44.606827 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.601342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.0.147.67:45519)
0809 12:07:44.601655 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.601660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.601684 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:07:44.601823 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.601830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.601959 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.602124 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:07:44.602483 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.603232 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.604638 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.605025 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.605124 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.605670 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:07:44.606168 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:07:44.606291 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.606441 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.606560 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:07:44.614977 24053 raft_consensus.cc:1261] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [term 1 FOLLOWER]: Refusing update from remote peer cdf243980d7d40a7bbae1d7d376a0566: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:07:44.614996 24245 raft_consensus.cc:1261] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [term 1 FOLLOWER]: Refusing update from remote peer cdf243980d7d40a7bbae1d7d376a0566: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:07:44.616076 24539 consensus_queue.cc:1035] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7cb995fc2b6456dbe4a7af289055d33" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 45469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:07:44.616444 24535 consensus_queue.cc:1035] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [LEADER]: Connected to new peer: Peer: permanent_uuid: "030d35048f184ebbacc138e11b415cf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 35553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:07:44.627835 24542 mvcc.cc:205] Tried to move back new op lower bound from 7316591675853086720 to 7316591675599597568. Current Snapshot: MvccSnapshot[applied={T|T < 7316591675853086720}]
I20260809 12:07:44.642880 23848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:07:44.643204 23848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:07:44.646826 23848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Sending DeleteTablet for 3 replicas of tablet 7efee9b586b54e23af11d89340928118
I20260809 12:07:44.648119 24217 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efee9b586b54e23af11d89340928118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:07:44 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:58910
I20260809 12:07:44.648149 24406 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efee9b586b54e23af11d89340928118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:07:44 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:51702
I20260809 12:07:44.648293 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 23907
I20260809 12:07:44.648909 24021 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efee9b586b54e23af11d89340928118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:07:44 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:41644
I20260809 12:07:44.649848 24556 tablet_replica.cc:330] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: stopping tablet replica
I20260809 12:07:44.650321 24556 raft_consensus.cc:2229] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:07:44.650997 24556 raft_consensus.cc:2258] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:07:44.653672 24556 ts_tablet_manager.cc:1905] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:07:44.662099 24556 ts_tablet_manager.cc:1918] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:07:44.662290 24556 log.cc:1196] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/7efee9b586b54e23af11d89340928118
I20260809 12:07:44.662796 24556 ts_tablet_manager.cc:1939] T 7efee9b586b54e23af11d89340928118 P cdf243980d7d40a7bbae1d7d376a0566: Deleting consensus metadata
I20260809 12:07:44.663815 24558 tablet_replica.cc:330] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: stopping tablet replica
I20260809 12:07:44.664364 24558 raft_consensus.cc:2229] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:07:44.664682 23829 catalog_manager.cc:4841] TS cdf243980d7d40a7bbae1d7d376a0566 (127.0.147.67:45519): tablet 7efee9b586b54e23af11d89340928118 (table test-table [id=db34fd5d29344beb926699d7aa9b2d0e]) successfully deleted
I20260809 12:07:44.665086 24558 raft_consensus.cc:2258] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:07:44.667078 24558 ts_tablet_manager.cc:1905] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:07:44.671680 24557 tablet_replica.cc:330] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: stopping tablet replica
I20260809 12:07:44.673097 24558 ts_tablet_manager.cc:1918] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:07:44.673197 24557 raft_consensus.cc:2229] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:07:44.673276 24558 log.cc:1196] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/7efee9b586b54e23af11d89340928118
I20260809 12:07:44.673758 24557 raft_consensus.cc:2258] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:07:44.673866 24558 ts_tablet_manager.cc:1939] T 7efee9b586b54e23af11d89340928118 P f7cb995fc2b6456dbe4a7af289055d33: Deleting consensus metadata
I20260809 12:07:44.674988 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.649203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.0.147.65:45469)
0809 12:07:44.649432 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.649441 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.649592 (+ 151us) server_negotiation.cc:407] Connection header received
0809 12:07:44.649817 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.649826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.649930 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.650051 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:44.652043 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.653667 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.655733 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.656687 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.656863 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.660329 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:44.660349 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:44.660373 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:44.660432 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:44.665569 (+ 5137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.670513 (+ 4944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.670538 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.670548 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.670810 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.672149 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.672157 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.672161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.672705 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:44.673034 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.674265 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.674557 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:07:44.675714 24110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:44.676142 23829 catalog_manager.cc:4841] TS f7cb995fc2b6456dbe4a7af289055d33 (127.0.147.65:45469): tablet 7efee9b586b54e23af11d89340928118 (table test-table [id=db34fd5d29344beb926699d7aa9b2d0e]) successfully deleted
I20260809 12:07:44.681345 24557 ts_tablet_manager.cc:1905] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:07:44.688607 24557 ts_tablet_manager.cc:1918] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:07:44.688784 24557 log.cc:1196] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/7efee9b586b54e23af11d89340928118
I20260809 12:07:44.689353 24557 ts_tablet_manager.cc:1939] T 7efee9b586b54e23af11d89340928118 P 030d35048f184ebbacc138e11b415cf8: Deleting consensus metadata
I20260809 12:07:44.691305 23831 catalog_manager.cc:4841] TS 030d35048f184ebbacc138e11b415cf8 (127.0.147.66:35553): tablet 7efee9b586b54e23af11d89340928118 (table test-table [id=db34fd5d29344beb926699d7aa9b2d0e]) successfully deleted
I20260809 12:07:44.770313 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 23907
I20260809 12:07:44.796193 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 24122
I20260809 12:07:44.812569 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.797077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58926 (local address 127.0.147.66:35553)
0809 12:07:44.797310 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.797316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.797339 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:07:44.797440 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.797446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.797549 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.797665 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:44.799090 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.800301 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.801661 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.802291 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.802437 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.805347 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:44.805365 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:44.805387 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:44.805430 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:44.809210 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.810083 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.810094 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.810097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.810276 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.810696 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.810702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.810707 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.811210 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:44.811516 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.811936 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.812181 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:07:44.813365 24304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:44.905946 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 24122
I20260809 12:07:44.942719 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 24317
I20260809 12:07:44.959705 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:07:44.943750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.0.147.67:45519)
0809 12:07:44.943983 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 12:07:44.943989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:07:44.944011 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:07:44.944083 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:07:44.944088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:07:44.944189 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:07:44.944324 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:07:44.945708 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.946984 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.948426 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:07:44.949152 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:07:44.949300 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:07:44.952181 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:07:44.952204 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:07:44.952225 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:07:44.952270 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:07:44.956487 (+ 4217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.957238 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.957248 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.957252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.957455 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:07:44.957869 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:07:44.957874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:07:44.957879 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:07:44.958353 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:07:44.958691 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 12:07:44.958874 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 12:07:44.959100 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:07:44.960525 24496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:45.085839 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 24317
I20260809 12:07:45.117570 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 23803
I20260809 12:07:45.118796 23896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:07:45.238730 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 23803
I20260809 12:08:05.265921 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:43625
--webserver_interface=127.0.147.126
--webserver_port=43635
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:43625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:05.622535 24572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:05.622825 24572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:05.622992 24572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:05.630187 24572 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 12:08:05.630275 24572 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 12:08:05.630359 24572 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 12:08:05.630450 24572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:05.630541 24572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:05.630669 24572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:05.630722 24572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:05.630815 24572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:05.640722 24572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:43625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:43625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=43635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:05.642026 24572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:05.643385 24572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:05.655864 24577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:05.655833 24578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:05.655802 24583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:05.656701 24572 server_base.cc:1034] running on GCE node
I20260809 12:08:05.657383 24572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:05.658628 24572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:05.659835 24572 hybrid_clock.cc:648] HybridClock initialized: now 1786277285659764 us; error 45 us; skew 500 ppm
I20260809 12:08:05.660161 24572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:05 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277285, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:05.666467 24572 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:08:05.668013 24572 webserver.cc:460] Webserver started at http://127.0.147.126:43635/ using document root <none> and password file <none>
I20260809 12:08:05.668704 24572 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:05.668823 24572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:05.674226 24572 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:08:05.677045 24592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:05.678313 24572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:08:05.678495 24572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8"
format_stamp: "Formatted at 2026-08-09 12:07:19 on dist-test-slave-sb2z"
server_key: "d6e08ed0a8817f31249c40a4d42f0014"
server_key_iv: "e20538a5b23e4b4f036430e125f11291"
server_key_version: "encryptionkey@0"
I20260809 12:08:05.679455 24572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:05.704882 24572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:05.708698 24572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:05.708978 24572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:05.727752 24572 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:43625
I20260809 12:08:05.727831 24657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:43625 every 8 connection(s)
I20260809 12:08:05.729238 24572 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:05.731949 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 24572
I20260809 12:08:05.733045 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:45469
--local_ip_for_outbound_sockets=127.0.147.65
--tserver_master_addrs=127.0.147.126:43625
--webserver_port=42129
--webserver_interface=127.0.147.65
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:08:05.738305 24659 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:08:05.743469 24659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Bootstrap starting.
I20260809 12:08:05.765769 24659 log.cc:824] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:05.782135 24659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:08:05.782720 24659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Bootstrap complete.
I20260809 12:08:05.792367 24659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:08:05.793282 24659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a45a9d2ff94c2c8f84a9151d88a6f8, State: Initialized, Role: FOLLOWER
I20260809 12:08:05.793864 24659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:08:05.794121 24659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:05.794252 24659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:05.794459 24659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 2 FOLLOWER]: Advancing to term 3
I20260809 12:08:05.798243 24659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:08:05.798724 24659 leader_election.cc:302] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32a45a9d2ff94c2c8f84a9151d88a6f8; no voters:
I20260809 12:08:05.799414 24659 leader_election.cc:288] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 12:08:05.799578 24663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 12:08:05.800987 24663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [term 3 LEADER]: Becoming Leader. State: Replica: 32a45a9d2ff94c2c8f84a9151d88a6f8, State: Running, Role: LEADER
I20260809 12:08:05.801533 24663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } }
I20260809 12:08:05.801854 24659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:05.804245 24664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } } }
I20260809 12:08:05.804679 24664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:05.808544 24665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32a45a9d2ff94c2c8f84a9151d88a6f8. Latest consensus state: current_term: 3 leader_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a45a9d2ff94c2c8f84a9151d88a6f8" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 43625 } } }
I20260809 12:08:05.808912 24665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:05.810551 24669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:05.819628 24669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:05.821159 24669 catalog_manager.cc:1259] Loaded cluster ID: 5d121e48a55b448dac144fcd384cb1d9
I20260809 12:08:05.821314 24669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:05.825495 24669 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:05.827140 24669 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Loaded TSK: 0
I20260809 12:08:05.827363 24669 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32a45a9d2ff94c2c8f84a9151d88a6f8: Loaded TSK: 1
F20260809 12:08:05.828761 24669 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4d586ea9ad google::LogMessage::Fail() at ??:0
@ 0x7f4d586ed88a google::LogMessage::SendToLog() at ??:0
@ 0x7f4d586ea4de google::LogMessage::Flush() at ??:0
I20260809 12:08:05.836876 24692 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f4d586eba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4d5c44e7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4d5c43d73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f4d6e1191c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f4d6e118783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4d6e1a42a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f4d6e1a40c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f4d6e1ee5e9 std::function<>::operator()() at ??:0
@ 0x7f4d6e11ccdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f4d6e11ac36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4d6e1a2610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f4d6e1a2439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d5b336345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4d5b33991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f4d5b339799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d5b314eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4d5e5ee6db start_thread at ??:0
@ 0x7f4d5758461f clone at ??:0
W20260809 12:08:06.147070 24661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:06.147329 24661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:06.147488 24661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:06.154657 24661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:06.154762 24661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:06.154881 24661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:08:06.164528 24661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:45469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=42129
--tserver_master_addrs=127.0.147.126:43625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:06.165688 24661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:06.167006 24661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:06.180064 24733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:06.181351 24735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:06.181365 24737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:06.182193 24661 server_base.cc:1034] running on GCE node
I20260809 12:08:06.182809 24661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:06.183959 24661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:06.185204 24661 hybrid_clock.cc:648] HybridClock initialized: now 1786277286185155 us; error 49 us; skew 500 ppm
I20260809 12:08:06.185585 24661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:06 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277286, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:06.188803 24661 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:08:06.190847 24661 webserver.cc:460] Webserver started at http://127.0.147.65:42129/ using document root <none> and password file <none>
I20260809 12:08:06.191586 24661 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:06.191747 24661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:06.197033 24661 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 12:08:06.199739 24747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:06.201083 24661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:08:06.201256 24661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "f7cb995fc2b6456dbe4a7af289055d33"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "ccd7b233da22769135738670803fb8b8"
server_key_iv: "522c25a6f39d4807c7dd11f5a38e354d"
server_key_version: "encryptionkey@0"
I20260809 12:08:06.202239 24661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:06.220366 24661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:06.224725 24661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:06.225010 24661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:06.226012 24661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:06.227362 24661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:06.227499 24661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:06.227696 24661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:06.227792 24661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:06.262512 24661 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:45469
I20260809 12:08:06.262603 24905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:45469 every 8 connection(s)
I20260809 12:08:06.264070 24661 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:08:06.268159 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 24661
I20260809 12:08:06.268824 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:35553
--local_ip_for_outbound_sockets=127.0.147.66
--tserver_master_addrs=127.0.147.126:43625
--webserver_port=40569
--webserver_interface=127.0.147.66
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:08:06.268922 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:06.266984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:38363)
0809 12:08:06.267779 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:06.268023 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":290,"threads_started":1}
W20260809 12:08:06.269675 24906 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:43625 (0 consecutive failures): Network error: Failed to ping master at 127.0.147.126:43625: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
I20260809 12:08:06.270483 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:06.270163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:34737)
0809 12:08:06.270311 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:06.270387 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 12:08:06.271559 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:06.271106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:35881)
0809 12:08:06.271267 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:06.271367 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 12:08:06.271811 24906 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:08:06.615361 24909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:06.615754 24909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:06.615942 24909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:06.623131 24909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:06.623287 24909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:06.623482 24909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:08:06.633172 24909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:35553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=40569
--tserver_master_addrs=127.0.147.126:43625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:06.634490 24909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:06.635869 24909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:06.649206 24917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:06.649430 24919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:06.649245 24915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:06.650069 24909 server_base.cc:1034] running on GCE node
I20260809 12:08:06.650674 24909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:06.651577 24909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:06.652794 24909 hybrid_clock.cc:648] HybridClock initialized: now 1786277286652745 us; error 44 us; skew 500 ppm
I20260809 12:08:06.653069 24909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:06 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277286, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:06.656380 24909 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:08:06.657775 24909 webserver.cc:460] Webserver started at http://127.0.147.66:40569/ using document root <none> and password file <none>
I20260809 12:08:06.658456 24909 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:06.658608 24909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:06.663980 24909 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:08:06.666543 24928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:06.667888 24909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:08:06.668068 24909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "030d35048f184ebbacc138e11b415cf8"
format_stamp: "Formatted at 2026-08-09 12:07:20 on dist-test-slave-sb2z"
server_key: "c209bfc9272956a244ebf9ff3b778685"
server_key_iv: "8740f224a76ffbeecb3a3c7fdba55ea3"
server_key_version: "encryptionkey@0"
I20260809 12:08:06.669047 24909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:06.682634 24909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:06.686553 24909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:06.686856 24909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:06.687882 24909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:06.689213 24909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:06.689343 24909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:06.689469 24909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:06.689566 24909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:06.724653 24909 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:35553
I20260809 12:08:06.724741 25091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:35553 every 8 connection(s)
I20260809 12:08:06.726112 24909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:08:06.730808 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 24909
I20260809 12:08:06.730849 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:06.729038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:52575)
0809 12:08:06.729744 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:06.729984 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":274,"threads_started":1}
W20260809 12:08:06.731684 25092 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:43625 (0 consecutive failures): Network error: Failed to ping master at 127.0.147.126:43625: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
I20260809 12:08:06.731671 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:45519
--local_ip_for_outbound_sockets=127.0.147.67
--tserver_master_addrs=127.0.147.126:43625
--webserver_port=40529
--webserver_interface=127.0.147.67
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:08:06.732571 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:06.732226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:59755)
0809 12:08:06.732383 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:06.732465 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 12:08:06.733520 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:06.733175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:46741)
0809 12:08:06.733315 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:06.733403 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:08:06.733824 25092 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:08:07.074898 25095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:07.075177 25095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:07.075332 25095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:07.082531 25095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:07.082638 25095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:07.082830 25095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:08:07.092608 25095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:45519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=40529
--tserver_master_addrs=127.0.147.126:43625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:07.093854 25095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:07.095175 25095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:07.108525 25104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:07.108600 25102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:07.108628 25101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:07.109625 25095 server_base.cc:1034] running on GCE node
I20260809 12:08:07.110248 25095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:07.111135 25095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:07.112362 25095 hybrid_clock.cc:648] HybridClock initialized: now 1786277287112327 us; error 48 us; skew 500 ppm
I20260809 12:08:07.112672 25095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:07 dist-test-slave-sb2z krb5kdc[22975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277287, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:07.118359 25095 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:08:07.119892 25095 webserver.cc:460] Webserver started at http://127.0.147.67:40529/ using document root <none> and password file <none>
I20260809 12:08:07.120590 25095 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:07.120743 25095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:07.125917 25095 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:08:07.128633 25112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:07.129845 25095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:08:07.130045 25095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "cdf243980d7d40a7bbae1d7d376a0566"
format_stamp: "Formatted at 2026-08-09 12:07:21 on dist-test-slave-sb2z"
server_key: "7f6f6ceca4a44db2348ec09830262d99"
server_key_iv: "83dfd9a6d851378ea8a0d182e724efb9"
server_key_version: "encryptionkey@0"
I20260809 12:08:07.130996 25095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:07.148182 25095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:07.152401 25095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:07.152676 25095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:07.153671 25095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:07.154943 25095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:07.155082 25095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:07.155206 25095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:07.155292 25095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:07.190030 25095 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:45519
I20260809 12:08:07.190863 25269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:45519 every 8 connection(s)
I20260809 12:08:07.192023 25095 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:08:07.194692 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 25095
I20260809 12:08:07.199123 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:07.195939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:51551)
0809 12:08:07.197308 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:07.197640 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":546,"threads_started":1}
W20260809 12:08:07.201934 25270 heartbeater.cc:644] Failed to heartbeat to 127.0.147.126:43625 (0 consecutive failures): Network error: Failed to ping master at 127.0.147.126:43625: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
I20260809 12:08:07.202921 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:07.202558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:42647)
0809 12:08:07.202712 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:07.202780 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 12:08:07.203956 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:07.203622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:33963)
0809 12:08:07.203761 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:07.203849 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:08:07.204210 25270 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 12:08:07.273226 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:07.272329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:53271)
0809 12:08:07.272829 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:07.272973 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":223,"threads_started":1}
I20260809 12:08:07.735306 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:07.734446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:54291)
0809 12:08:07.734890 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:07.735041 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":213,"threads_started":1}
I20260809 12:08:08.205523 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:08.204718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:48261)
0809 12:08:08.205202 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:08.205299 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":234,"threads_started":1}
I20260809 12:08:08.275017 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:08.274215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:41235)
0809 12:08:08.274738 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:08.274828 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":264,"threads_started":1}
I20260809 12:08:08.737000 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:08.736307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:54341)
0809 12:08:08.736761 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:08.736835 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":217,"threads_started":1}
I20260809 12:08:09.207381 25279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:09.206410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:55767)
0809 12:08:09.207059 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:09.207141 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":347,"threads_started":1}
I20260809 12:08:09.276684 25280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:09.276002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:44733)
0809 12:08:09.276436 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:09.276518 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":229,"threads_started":1}
I20260809 12:08:09.738531 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:09.737839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:51983)
0809 12:08:09.738275 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:09.738352 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":240,"threads_started":1}
I20260809 12:08:10.208993 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:10.208277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:51525)
0809 12:08:10.208742 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:10.208821 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":254,"threads_started":1}
I20260809 12:08:10.278307 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:10.277576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:51543)
0809 12:08:10.278026 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:10.278116 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":231,"threads_started":1}
I20260809 12:08:10.740173 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:10.739391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:33485)
0809 12:08:10.739892 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:10.739976 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":283,"threads_started":1}
I20260809 12:08:11.210585 25288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:11.209877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:57379)
0809 12:08:11.210314 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:11.210393 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":226,"threads_started":1}
I20260809 12:08:11.280045 25289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:11.279289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:57861)
0809 12:08:11.279795 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:11.279875 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":291,"threads_started":1}
I20260809 12:08:11.741820 25290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:11.741103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:40785)
0809 12:08:11.741557 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:11.741651 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":248,"threads_started":1}
I20260809 12:08:12.212291 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:12.211516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:57725)
0809 12:08:12.212045 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:12.212125 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":226,"threads_started":1}
I20260809 12:08:12.281594 25292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:12.280882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:44821)
0809 12:08:12.281343 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:12.281431 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":238,"threads_started":1}
I20260809 12:08:12.743402 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:12.742698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:57887)
0809 12:08:12.743144 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:12.743234 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":232,"threads_started":1}
I20260809 12:08:13.213845 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:13.213129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:58537)
0809 12:08:13.213590 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:13.213679 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":237,"threads_started":1}
I20260809 12:08:13.283419 25295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:13.282547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:34011)
0809 12:08:13.283108 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:13.283199 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":243,"threads_started":1}
I20260809 12:08:13.745131 25296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:13.744323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:38809)
0809 12:08:13.744787 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:13.744888 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":212,"threads_started":1}
I20260809 12:08:14.215600 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:14.214741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:48683)
0809 12:08:14.215251 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:14.215333 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":234,"threads_started":1}
I20260809 12:08:14.285014 25299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:14.284318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:50901)
0809 12:08:14.284751 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:14.284830 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
I20260809 12:08:14.746809 25300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:14.746050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:44595)
0809 12:08:14.746563 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:14.746646 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":257,"threads_started":1}
I20260809 12:08:15.217298 25301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:15.216593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:34555)
0809 12:08:15.217043 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:15.217128 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":230,"threads_started":1}
I20260809 12:08:15.286661 25302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:15.285873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:34989)
0809 12:08:15.286350 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:15.286429 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":248,"threads_started":1}
I20260809 12:08:15.748533 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:15.747803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:45933)
0809 12:08:15.748261 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:15.748340 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":227,"threads_started":1}
I20260809 12:08:16.219079 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:16.218282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:33789)
0809 12:08:16.218796 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:16.218877 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":275,"threads_started":1}
I20260809 12:08:16.288255 25307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:16.287590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:51579)
0809 12:08:16.288023 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:16.288100 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":236,"threads_started":1}
I20260809 12:08:16.750136 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:16.749428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:60925)
0809 12:08:16.749878 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:16.749955 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":258,"threads_started":1}
I20260809 12:08:17.220686 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:17.220003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:50887)
0809 12:08:17.220426 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:17.220503 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":232,"threads_started":1}
I20260809 12:08:17.290221 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:17.289238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:33879)
0809 12:08:17.289802 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:17.289940 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":253,"threads_started":1}
I20260809 12:08:17.751904 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:17.751115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:44413)
0809 12:08:17.751631 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:17.751715 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":214,"threads_started":1}
I20260809 12:08:18.222364 25314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:18.221624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:46445)
0809 12:08:18.222079 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:18.222173 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":215,"threads_started":1}
I20260809 12:08:18.291790 25316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:18.291114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:34059)
0809 12:08:18.291546 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:18.291637 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":221,"threads_started":1}
I20260809 12:08:18.753458 25318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:18.752777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:37153)
0809 12:08:18.753228 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:18.753304 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":237,"threads_started":1}
I20260809 12:08:19.224045 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:19.223294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:44553)
0809 12:08:19.223795 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:19.223880 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":292,"threads_started":1}
I20260809 12:08:19.293434 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:19.292647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:59093)
0809 12:08:19.293144 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:19.293241 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":236,"threads_started":1}
I20260809 12:08:19.755402 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:19.754570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:43429)
0809 12:08:19.755128 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:19.755224 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":304,"threads_started":1}
I20260809 12:08:20.225769 25323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:20.224974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:49713)
0809 12:08:20.225469 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:20.225574 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":252,"threads_started":1}
I20260809 12:08:20.295331 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:20.294379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:46061)
0809 12:08:20.294962 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:20.295090 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":320,"threads_started":1}
I20260809 12:08:20.757302 25330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:20.756416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:32859)
0809 12:08:20.756978 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:20.757100 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":283,"threads_started":1}
I20260809 12:08:21.227418 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:21.226696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:59547)
0809 12:08:21.227143 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:21.227237 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":225,"threads_started":1}
I20260809 12:08:21.297369 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:21.296453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.65:45471)
0809 12:08:21.297094 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:21.297180 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":463,"threads_started":1}
I20260809 12:08:21.759104 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:21.758375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.66:39457)
0809 12:08:21.758814 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:21.758909 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":232,"threads_started":1}
I20260809 12:08:22.206241 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 24661
I20260809 12:08:22.224350 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:22.207488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48100 (local address 127.0.147.65:45469)
0809 12:08:22.208073 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:22.208088 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:22.208128 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:08:22.209633 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:22.209642 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:22.210074 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:22.210417 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:22.211812 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:22.213079 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:22.214347 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:22.215052 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:22.215187 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:22.218023 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:22.218030 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:22.218046 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:22.218087 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:22.221265 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:22.221987 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:22.221994 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:22.222000 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:22.222167 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:22.222516 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:22.222519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:22.222521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:22.223028 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:22.223256 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:22.223737 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 12:08:22.224072 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":262,"threads_started":1}
I20260809 12:08:22.225791 24900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:22.229693 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:22.228618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:43625 (local address 127.0.147.67:58919)
0809 12:08:22.229242 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:22.229386 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.126:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":292,"threads_started":1}
I20260809 12:08:22.310565 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 24661
I20260809 12:08:22.339869 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 24909
I20260809 12:08:22.360262 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:22.340949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.0.147.66:35553)
0809 12:08:22.341476 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:22.341493 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:22.341535 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:08:22.343037 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:22.343048 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:22.343477 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:22.343849 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:22.345269 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:22.346488 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:22.347847 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:22.348561 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:22.348707 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:22.353849 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:22.353858 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:22.353875 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:22.353959 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:22.357006 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:22.357697 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:22.357703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:22.357716 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:22.357906 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:22.358272 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:22.358275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:22.358277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:22.358799 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:22.359073 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:22.359403 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 12:08:22.359962 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":251,"threads_started":1}
I20260809 12:08:22.361673 25084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:22.446731 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 24909
I20260809 12:08:22.473790 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 25095
I20260809 12:08:22.491225 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:22.474854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59254 (local address 127.0.147.67:45519)
0809 12:08:22.475452 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:22.475467 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:22.475509 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:08:22.477233 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:22.477244 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:22.477667 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:22.477956 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:22.479254 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:22.480697 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:22.481954 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:22.482598 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:22.482728 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:22.485396 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:22.485404 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:22.485418 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:22.485453 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:22.488399 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:22.489003 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:22.489009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:22.489015 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:22.489188 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:22.489524 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:22.489527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:22.489529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:22.490018 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:22.490250 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:22.490591 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 12:08:22.490899 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":291,"threads_started":1}
I20260809 12:08:22.492811 25264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:22.577454 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 25095
2026-08-09T12:08:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65054 ms)
[----------] 30 tests from SecurityITest (312241 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:08:22 dist-test-slave-sb2z krb5kdc[25348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:08:22 dist-test-slave-sb2z krb5kdc[25348](info): set up 2 sockets
Aug 09 12:08:22 dist-test-slave-sb2z krb5kdc[25348](info): commencing operation
krb5kdc: starting...
W20260809 12:08:23.898864 589 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:08:24 dist-test-slave-sb2z krb5kdc[25348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:08:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:08:24.243131 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38103
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:38103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:24.590584 25364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:24.590879 25364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:24.591061 25364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:24.598424 25364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:24.598564 25364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:24.598656 25364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:24.598769 25364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:24.598852 25364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:24.608681 25364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:38103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:38103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:24.610020 25364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:24.611330 25364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:24.623129 25375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:24.623211 25372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:24.623179 25373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:24.624430 25364 server_base.cc:1034] running on GCE node
I20260809 12:08:24.625339 25364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:24.626734 25364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:24.627972 25364 hybrid_clock.cc:648] HybridClock initialized: now 1786277304627926 us; error 54 us; skew 500 ppm
I20260809 12:08:24.628332 25364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:24 dist-test-slave-sb2z krb5kdc[25348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277304, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:24.631415 25364 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:08:24.633021 25364 webserver.cc:460] Webserver started at http://127.0.147.126:43203/ using document root <none> and password file <none>
I20260809 12:08:24.633700 25364 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:24.633852 25364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:24.634186 25364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:24.637213 25364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "cd9c7ca213c14676a640dc472505981b"
format_stamp: "Formatted at 2026-08-09 12:08:24 on dist-test-slave-sb2z"
server_key: "7f53aa62a1e78177b19e41f484538aed"
server_key_iv: "d7a2cf4b341e70b8bc60bb88074b7bdc"
server_key_version: "encryptionkey@0"
I20260809 12:08:24.637977 25364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "cd9c7ca213c14676a640dc472505981b"
format_stamp: "Formatted at 2026-08-09 12:08:24 on dist-test-slave-sb2z"
server_key: "7f53aa62a1e78177b19e41f484538aed"
server_key_iv: "d7a2cf4b341e70b8bc60bb88074b7bdc"
server_key_version: "encryptionkey@0"
I20260809 12:08:24.643157 25364 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:08:24.646517 25385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:24.647867 25364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:08:24.648063 25364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "cd9c7ca213c14676a640dc472505981b"
format_stamp: "Formatted at 2026-08-09 12:08:24 on dist-test-slave-sb2z"
server_key: "7f53aa62a1e78177b19e41f484538aed"
server_key_iv: "d7a2cf4b341e70b8bc60bb88074b7bdc"
server_key_version: "encryptionkey@0"
I20260809 12:08:24.648262 25364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:24.667928 25364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:24.671830 25364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:24.672122 25364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:24.691025 25364 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:38103
I20260809 12:08:24.691053 25447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:38103 every 8 connection(s)
I20260809 12:08:24.692415 25364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:24.693516 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 25364
I20260809 12:08:24.693784 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:24.694638 589 external_mini_cluster.cc:1428] Setting key 557980488bcdab5d9bb46bdeae79a0c7
I20260809 12:08:24.697257 25448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:08:24 dist-test-slave-sb2z krb5kdc[25348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:24.710613 25448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b: Bootstrap starting.
I20260809 12:08:24.714815 25448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:24.716180 25448 log.cc:824] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:24.719417 25448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b: No bootstrap required, opened a new log
I20260809 12:08:24.719712 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:24.696773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.0.147.126:38103)
0809 12:08:24.697615 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:24.697626 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:24.697667 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:08:24.699329 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:24.699350 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:24.699799 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:24.700142 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:24.702064 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:24.703387 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:24.705312 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:24.705949 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:24.706089 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:24.711245 (+ 5156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:24.711255 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:24.711271 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:24.711321 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:24.715305 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:24.716047 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:24.716055 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:24.716063 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:24.716287 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:24.716750 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:24.716758 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:24.716763 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:24.717328 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:24.717658 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:24.717971 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 12:08:24.718376 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":401,"threads_started":1}
I20260809 12:08:24.726954 25448 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9c7ca213c14676a640dc472505981b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38103 } }
I20260809 12:08:24.727304 25448 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:24.727381 25448 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd9c7ca213c14676a640dc472505981b, State: Initialized, Role: FOLLOWER
I20260809 12:08:24.727949 25448 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9c7ca213c14676a640dc472505981b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38103 } }
I20260809 12:08:24.728207 25448 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:24.728391 25448 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:24.728592 25448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:24.730811 25448 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9c7ca213c14676a640dc472505981b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38103 } }
I20260809 12:08:24.731314 25448 leader_election.cc:302] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd9c7ca213c14676a640dc472505981b; no voters:
I20260809 12:08:24.731961 25448 leader_election.cc:288] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:24.732147 25455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:24.732838 25455 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [term 1 LEADER]: Becoming Leader. State: Replica: cd9c7ca213c14676a640dc472505981b, State: Running, Role: LEADER
I20260809 12:08:24.733383 25455 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9c7ca213c14676a640dc472505981b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38103 } }
I20260809 12:08:24.734112 25448 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:24.736364 25457 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd9c7ca213c14676a640dc472505981b. Latest consensus state: current_term: 1 leader_uuid: "cd9c7ca213c14676a640dc472505981b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9c7ca213c14676a640dc472505981b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38103 } } }
I20260809 12:08:24.736924 25457 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:24.736860 25456 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd9c7ca213c14676a640dc472505981b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9c7ca213c14676a640dc472505981b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 38103 } } }
I20260809 12:08:24.737195 25456 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:24.738036 25463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:24.743782 25463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:24.752684 25463 catalog_manager.cc:1347] Generated new cluster ID: 2129927834994e1eb103a3d61d6c4e06
I20260809 12:08:24.752810 25463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:24.762591 25463 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:24.764189 25463 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:24.779985 25463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b: Generated new TSK 0
I20260809 12:08:24.780929 25463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:08:24.875631 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38103
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:25.233788 25483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:25.234091 25483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:25.234200 25483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:25.242640 25483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:25.242756 25483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:25.242883 25483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:08:25.253545 25483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:25.255179 25483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:25.256712 25483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:25.270087 25491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:25.270144 25492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:25.270936 25494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:25.271279 25483 server_base.cc:1034] running on GCE node
I20260809 12:08:25.272221 25483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:25.273231 25483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:25.274468 25483 hybrid_clock.cc:648] HybridClock initialized: now 1786277305274419 us; error 39 us; skew 500 ppm
I20260809 12:08:25.274797 25483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:25 dist-test-slave-sb2z krb5kdc[25348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277305, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:25.278067 25483 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:08:25.279647 25483 webserver.cc:460] Webserver started at http://127.0.147.65:42299/ using document root <none> and password file <none>
I20260809 12:08:25.280294 25483 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:25.280392 25483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:25.280655 25483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:25.283571 25483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "970fae78fc5e4191966c340e18f97663"
format_stamp: "Formatted at 2026-08-09 12:08:25 on dist-test-slave-sb2z"
server_key: "05d96c987b971827372a3e030e2ef136"
server_key_iv: "d1471c39bb2e2bc6f3adaca6fadf35fb"
server_key_version: "encryptionkey@0"
I20260809 12:08:25.284320 25483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "970fae78fc5e4191966c340e18f97663"
format_stamp: "Formatted at 2026-08-09 12:08:25 on dist-test-slave-sb2z"
server_key: "05d96c987b971827372a3e030e2ef136"
server_key_iv: "d1471c39bb2e2bc6f3adaca6fadf35fb"
server_key_version: "encryptionkey@0"
I20260809 12:08:25.289124 25483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 12:08:25.292394 25502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:25.293645 25483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:08:25.293818 25483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "970fae78fc5e4191966c340e18f97663"
format_stamp: "Formatted at 2026-08-09 12:08:25 on dist-test-slave-sb2z"
server_key: "05d96c987b971827372a3e030e2ef136"
server_key_iv: "d1471c39bb2e2bc6f3adaca6fadf35fb"
server_key_version: "encryptionkey@0"
I20260809 12:08:25.294010 25483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:25.311581 25483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:25.315330 25483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:25.315676 25483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:25.316694 25483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:25.320264 25483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:25.320367 25483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:25.320526 25483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:25.320585 25483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:25.355690 25483 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:40443
I20260809 12:08:25.355722 25659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:40443 every 8 connection(s)
I20260809 12:08:25.356971 25483 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:08:25.363757 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 25483
I20260809 12:08:25.363977 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:08:25.364812 589 external_mini_cluster.cc:1428] Setting key 2ff346b251bd320d1d0014292404db1c
Aug 09 12:08:25 dist-test-slave-sb2z krb5kdc[25348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277305, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:25.378453 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:25.359584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:38771 (local address 127.0.147.126:38103)
0809 12:08:25.360071 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:25.360078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:25.361018 (+ 940us) server_negotiation.cc:407] Connection header received
0809 12:08:25.362798 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:25.362802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:25.362869 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:25.362965 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:25.365037 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:25.365886 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:25.367820 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:25.368505 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:25.368658 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:25.372612 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:25.372617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:25.372620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:25.372651 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:25.375207 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:25.376171 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:25.376175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:25.376177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:25.376293 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:25.376776 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:25.376779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:25.376781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:25.377037 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:25.377147 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:25.378019 (+ 872us) server_negotiation.cc:299] Negotiation successful
0809 12:08:25.378189 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260809 12:08:25.379908 25663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:25.360163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38103 (local address 127.0.147.65:38771)
0809 12:08:25.360840 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:25.360884 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:25.362556 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:25.363183 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:25.363190 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:25.364047 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:25.364853 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:25.364883 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:25.366040 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:25.366045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:25.367680 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:25.367693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:25.368717 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:25.368722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:25.368906 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:25.369599 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:08:25.369625 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:25.372366 (+ 2741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:25.375351 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:25.375362 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:25.375379 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:25.376049 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:25.376400 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:25.376407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:25.376414 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:25.376663 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:25.377166 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:25.377177 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:25.377619 (+ 442us) client_negotiation.cc:769] Sending connection context
0809 12:08:25.377920 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 12:08:25.378479 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":261,"threads_started":1}
I20260809 12:08:25.381532 25660 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38103
I20260809 12:08:25.381845 25660 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:25.382550 25660 heartbeater.cc:511] Master 127.0.147.126:38103 requested a full tablet report, sending...
I20260809 12:08:25.384644 25405 ts_manager.cc:194] Registered new tserver with Master: 970fae78fc5e4191966c340e18f97663 (127.0.147.65:40443)
I20260809 12:08:25.386116 25405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:38771
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:08:25.436806 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38103
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:25.782264 25667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:25.782588 25667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:25.782761 25667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:25.789923 25667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:25.790086 25667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:25.790266 25667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:08:25.799799 25667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:25.801035 25667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:25.802345 25667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:25.815303 25681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:25.815303 25676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:25.815346 25684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:25.816175 25667 server_base.cc:1034] running on GCE node
I20260809 12:08:25.816766 25667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:25.817751 25667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:25.818975 25667 hybrid_clock.cc:648] HybridClock initialized: now 1786277305818919 us; error 43 us; skew 500 ppm
I20260809 12:08:25.819294 25667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:25 dist-test-slave-sb2z krb5kdc[25348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277305, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:25.822633 25667 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:08:25.824041 25667 webserver.cc:460] Webserver started at http://127.0.147.66:34099/ using document root <none> and password file <none>
I20260809 12:08:25.824648 25667 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:25.824748 25667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:25.825002 25667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:25.827976 25667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "404b8bbb0ead48be841f47c85c4f79c7"
format_stamp: "Formatted at 2026-08-09 12:08:25 on dist-test-slave-sb2z"
server_key: "dbd98a367ceedf1e35a1ad49306d0945"
server_key_iv: "ecbca1be5234b1f4777d12602ca07fc3"
server_key_version: "encryptionkey@0"
I20260809 12:08:25.828714 25667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "404b8bbb0ead48be841f47c85c4f79c7"
format_stamp: "Formatted at 2026-08-09 12:08:25 on dist-test-slave-sb2z"
server_key: "dbd98a367ceedf1e35a1ad49306d0945"
server_key_iv: "ecbca1be5234b1f4777d12602ca07fc3"
server_key_version: "encryptionkey@0"
I20260809 12:08:25.833585 25667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 12:08:25.837181 25691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:25.838428 25667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:08:25.838591 25667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "404b8bbb0ead48be841f47c85c4f79c7"
format_stamp: "Formatted at 2026-08-09 12:08:25 on dist-test-slave-sb2z"
server_key: "dbd98a367ceedf1e35a1ad49306d0945"
server_key_iv: "ecbca1be5234b1f4777d12602ca07fc3"
server_key_version: "encryptionkey@0"
I20260809 12:08:25.838784 25667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:25.851508 25667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:25.855999 25667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:25.856299 25667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:25.857227 25667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:25.858659 25667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:25.858752 25667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:25.858922 25667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:25.859001 25667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:25.893654 25667 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:46303
I20260809 12:08:25.893759 25845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:46303 every 8 connection(s)
I20260809 12:08:25.894919 25667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:08:25.898639 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 25667
I20260809 12:08:25.898883 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:08:25.899549 589 external_mini_cluster.cc:1428] Setting key f1f3a01c56c4f5341f8b87631a47236f
Aug 09 12:08:25 dist-test-slave-sb2z krb5kdc[25348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277305, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:25.919981 25851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:25.897564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:43409 (local address 127.0.147.126:38103)
0809 12:08:25.903976 (+ 6412us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:25.903981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:25.903998 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:08:25.904063 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:25.904069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:25.904154 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:25.904293 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:25.906026 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:25.906758 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:25.908405 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:25.908852 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:25.908991 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:25.912810 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:25.912819 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:25.912826 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:25.912873 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:25.916590 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:25.917324 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:25.917331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:25.917337 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:25.917517 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:25.917949 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:25.917956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:25.917962 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:25.918356 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:25.918527 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:25.919191 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 12:08:25.919451 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6268,"thread_start_us":6161,"threads_started":1}
I20260809 12:08:25.920655 25852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:25.898277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38103 (local address 127.0.147.66:43409)
0809 12:08:25.899316 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:25.899375 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:25.901290 (+ 1915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:25.904536 (+ 3246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:25.904545 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:25.905255 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:25.905822 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:25.905846 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:25.906926 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:25.906930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:25.908252 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:25.908265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:25.909108 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:25.909116 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:25.909344 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:25.910054 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:08:25.910086 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:25.912536 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:25.916734 (+ 4198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:25.916742 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:25.916756 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:25.917186 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:25.917630 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:25.917633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:25.917635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:25.917829 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:25.918481 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:25.918487 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:25.918804 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 12:08:25.919033 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 12:08:25.919428 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":457,"threads_started":1}
I20260809 12:08:25.922279 25846 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38103
I20260809 12:08:25.922627 25846 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:25.923357 25846 heartbeater.cc:511] Master 127.0.147.126:38103 requested a full tablet report, sending...
I20260809 12:08:25.924943 25405 ts_manager.cc:194] Registered new tserver with Master: 404b8bbb0ead48be841f47c85c4f79c7 (127.0.147.66:46303)
I20260809 12:08:25.925806 25405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:43409
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:08:25.977123 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38103
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:26.317860 25857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:26.318173 25857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:26.318349 25857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:26.326088 25857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:26.326234 25857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:26.326398 25857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:08:26.336151 25857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:38103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:26.337389 25857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:26.338696 25857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:26.352021 25875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:26.352082 25871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:26.352087 25872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:26.353313 25857 server_base.cc:1034] running on GCE node
I20260809 12:08:26.353850 25857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:26.354918 25857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:26.356180 25857 hybrid_clock.cc:648] HybridClock initialized: now 1786277306356131 us; error 41 us; skew 500 ppm
I20260809 12:08:26.356513 25857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:26 dist-test-slave-sb2z krb5kdc[25348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277306, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:26.359956 25857 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:08:26.361496 25857 webserver.cc:460] Webserver started at http://127.0.147.67:44575/ using document root <none> and password file <none>
I20260809 12:08:26.362166 25857 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:26.362316 25857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:26.362620 25857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:26.365420 25857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "9edddbbadbad465b8925d15bb4f5798f"
format_stamp: "Formatted at 2026-08-09 12:08:26 on dist-test-slave-sb2z"
server_key: "917a0fc54e1edce8b5954d2eba15f829"
server_key_iv: "711e387b5913d37bc63735f25a66086c"
server_key_version: "encryptionkey@0"
I20260809 12:08:26.366248 25857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "9edddbbadbad465b8925d15bb4f5798f"
format_stamp: "Formatted at 2026-08-09 12:08:26 on dist-test-slave-sb2z"
server_key: "917a0fc54e1edce8b5954d2eba15f829"
server_key_iv: "711e387b5913d37bc63735f25a66086c"
server_key_version: "encryptionkey@0"
I20260809 12:08:26.371220 25857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:08:26.374660 25884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:26.375999 25857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:08:26.376185 25857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "9edddbbadbad465b8925d15bb4f5798f"
format_stamp: "Formatted at 2026-08-09 12:08:26 on dist-test-slave-sb2z"
server_key: "917a0fc54e1edce8b5954d2eba15f829"
server_key_iv: "711e387b5913d37bc63735f25a66086c"
server_key_version: "encryptionkey@0"
I20260809 12:08:26.376385 25857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:26.389628 25660 heartbeater.cc:503] Master 127.0.147.126:38103 was elected leader, sending a full tablet report...
I20260809 12:08:26.401512 25857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:26.405848 25857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:26.406160 25857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:26.407131 25857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:26.408525 25857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:26.408632 25857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:26.408785 25857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:26.408861 25857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:26.443804 25857 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:46883
I20260809 12:08:26.443859 26037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:46883 every 8 connection(s)
I20260809 12:08:26.445122 25857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:08:26.449748 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 25857
I20260809 12:08:26.449959 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:08:26.450501 589 external_mini_cluster.cc:1428] Setting key bb5025ef6434f6c29fbf6704903fd203
Aug 09 12:08:26 dist-test-slave-sb2z krb5kdc[25348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277306, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:26.466215 26042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.447653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:56437 (local address 127.0.147.126:38103)
0809 12:08:26.448111 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.448116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.448944 (+ 828us) server_negotiation.cc:407] Connection header received
0809 12:08:26.451605 (+ 2661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.451611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.451697 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.451831 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:26.453644 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.454426 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.456070 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.456467 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.456570 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.459945 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:26.459949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:26.459952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:26.459984 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:26.462498 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:26.463184 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:26.463188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:26.463190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:26.463308 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:26.463970 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:26.463973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:26.463975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:26.464289 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:26.464444 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.465452 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.465720 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":245,"threads_started":1}
I20260809 12:08:26.467470 26043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.448084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:38103 (local address 127.0.147.67:56437)
0809 12:08:26.448782 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:26.448818 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:26.451191 (+ 2373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:26.452230 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:26.452242 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:26.452904 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:26.453466 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.453485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.454587 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.454591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.455930 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.455943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.456672 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.456677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.456866 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.457487 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:08:26.457515 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:26.459747 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:26.462634 (+ 2887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:26.462641 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:26.462652 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:26.463074 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:26.463450 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:26.463461 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:26.463463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:26.463800 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:26.464480 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:26.464489 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:26.464954 (+ 465us) client_negotiation.cc:769] Sending connection context
0809 12:08:26.465305 (+ 351us) client_negotiation.cc:240] Negotiation successful
0809 12:08:26.465909 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":274,"threads_started":1}
I20260809 12:08:26.468869 26040 heartbeater.cc:347] Connected to a master server at 127.0.147.126:38103
I20260809 12:08:26.469158 26040 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:26.469851 26040 heartbeater.cc:511] Master 127.0.147.126:38103 requested a full tablet report, sending...
I20260809 12:08:26.471251 25405 ts_manager.cc:194] Registered new tserver with Master: 9edddbbadbad465b8925d15bb4f5798f (127.0.147.67:46883)
I20260809 12:08:26.472091 25405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:56437
I20260809 12:08:26.478245 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:08:26.495191 26042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.482972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.0.147.126:38103)
0809 12:08:26.483146 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.483150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.483735 (+ 585us) server_negotiation.cc:407] Connection header received
0809 12:08:26.483863 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.483867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.483925 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.484002 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:26.485447 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.486147 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.487456 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.488006 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.488107 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.490575 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:26.490579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:26.490582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:26.490609 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:26.492978 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:26.493585 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:26.493588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:26.493590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:26.493703 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:26.494016 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:26.494019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:26.494021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:26.494329 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:26.494509 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.494800 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.494956 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:08:26.498975 25405 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:08:26.505900 25405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:08:26.536996 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.518478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.0.147.65:40443)
0809 12:08:26.519262 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.519273 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.521066 (+ 1793us) server_negotiation.cc:407] Connection header received
0809 12:08:26.521175 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.521186 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.521447 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.521660 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:26.522469 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.530063 (+ 7594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.533549 (+ 3486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.535304 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.535453 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.535744 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.536464 (+ 720us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.536670 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":367,"threads_started":1}
I20260809 12:08:26.537038 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.518165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40443 (local address 127.0.0.1:51882)
0809 12:08:26.520593 (+ 2428us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:26.520640 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:26.520846 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:26.521950 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:26.521959 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:26.521990 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:26.522305 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.522318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.530222 (+ 7904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.530232 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.533380 (+ 3148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.533393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.535922 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.535929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.536190 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.536245 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 12:08:26.536393 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 12:08:26.536606 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2175,"thread_start_us":1999,"threads_started":1}
I20260809 12:08:26.541239 25565 tablet_service.cc:1467] Processing CreateTablet for tablet dab67988b9e640adaadfb2d786ac35ff (DEFAULT_TABLE table=test-table [id=27d6ce44c5c54a2f812017dbd3b13c26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:26.542790 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.523780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:46883 (local address 127.0.0.1:43048)
0809 12:08:26.533438 (+ 9658us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:26.533463 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:26.533614 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:26.535359 (+ 1745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:26.535367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:26.535401 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:26.535898 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.535911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.537836 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.537841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.539964 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.539979 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.541411 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.541417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.541836 (+ 419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.541897 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 12:08:26.542082 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 12:08:26.542339 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9481,"mutex_wait_us":84,"thread_start_us":291,"threads_started":1}
I20260809 12:08:26.542821 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.521366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.0.147.67:46883)
0809 12:08:26.534312 (+ 12946us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.534325 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.534369 (+ 44us) server_negotiation.cc:407] Connection header received
0809 12:08:26.534753 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.534766 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.535110 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.535362 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:26.536119 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.537688 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.540189 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.541180 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.541336 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.541574 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.542316 (+ 742us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.542496 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12706,"thread_start_us":13057,"threads_started":1}
I20260809 12:08:26.542902 25565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab67988b9e640adaadfb2d786ac35ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:26.544641 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.523981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.0.147.66:46303)
0809 12:08:26.536016 (+ 12035us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.536027 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.536057 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:08:26.536158 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.536166 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.536394 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.537431 (+ 1037us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:26.537638 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.539267 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.542094 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.543507 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.543713 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.543895 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.544164 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.544340 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11867,"thread_start_us":12563,"threads_started":1}
I20260809 12:08:26.544644 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.520497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:46303 (local address 127.0.0.1:56234)
0809 12:08:26.524408 (+ 3911us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:26.524433 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:26.524584 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:26.536806 (+ 12222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:26.536811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:26.536838 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:26.537232 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.537245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.539415 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.539420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.541899 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.541917 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.543800 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.543806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.543969 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.543998 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 12:08:26.544106 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 12:08:26.544248 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3745,"mutex_wait_us":160,"thread_start_us":12556,"threads_started":1}
I20260809 12:08:26.546902 25947 tablet_service.cc:1467] Processing CreateTablet for tablet dab67988b9e640adaadfb2d786ac35ff (DEFAULT_TABLE table=test-table [id=27d6ce44c5c54a2f812017dbd3b13c26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:26.548148 25947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab67988b9e640adaadfb2d786ac35ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:26.548990 25756 tablet_service.cc:1467] Processing CreateTablet for tablet dab67988b9e640adaadfb2d786ac35ff (DEFAULT_TABLE table=test-table [id=27d6ce44c5c54a2f812017dbd3b13c26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:26.550392 25756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab67988b9e640adaadfb2d786ac35ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:26.558694 26065 tablet_bootstrap.cc:492] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: Bootstrap starting.
I20260809 12:08:26.560606 26066 tablet_bootstrap.cc:492] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: Bootstrap starting.
I20260809 12:08:26.562467 26065 tablet_bootstrap.cc:654] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:26.563465 26065 log.cc:824] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:26.564325 26066 tablet_bootstrap.cc:654] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:26.565366 26068 tablet_bootstrap.cc:492] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: Bootstrap starting.
I20260809 12:08:26.565620 26066 log.cc:824] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:26.568647 26066 tablet_bootstrap.cc:492] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: No bootstrap required, opened a new log
I20260809 12:08:26.568907 26066 ts_tablet_manager.cc:1397] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260809 12:08:26.569811 26068 tablet_bootstrap.cc:654] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:26.571341 26068 log.cc:824] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:26.574581 26065 tablet_bootstrap.cc:492] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: No bootstrap required, opened a new log
I20260809 12:08:26.574874 26065 ts_tablet_manager.cc:1397] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260809 12:08:26.575184 26068 tablet_bootstrap.cc:492] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: No bootstrap required, opened a new log
I20260809 12:08:26.575441 26068 ts_tablet_manager.cc:1397] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 12:08:26.576540 26066 raft_consensus.cc:348] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } }
I20260809 12:08:26.576890 26066 raft_consensus.cc:374] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:26.577024 26066 raft_consensus.cc:729] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edddbbadbad465b8925d15bb4f5798f, State: Initialized, Role: FOLLOWER
I20260809 12:08:26.577626 26066 consensus_queue.cc:260] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } }
I20260809 12:08:26.578722 26040 heartbeater.cc:503] Master 127.0.147.126:38103 was elected leader, sending a full tablet report...
I20260809 12:08:26.579377 26066 ts_tablet_manager.cc:1428] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260809 12:08:26.585718 26065 raft_consensus.cc:348] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } }
I20260809 12:08:26.586172 26065 raft_consensus.cc:374] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:26.586293 26065 raft_consensus.cc:729] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 970fae78fc5e4191966c340e18f97663, State: Initialized, Role: FOLLOWER
I20260809 12:08:26.586218 26068 raft_consensus.cc:348] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } }
I20260809 12:08:26.586650 26068 raft_consensus.cc:374] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:26.586752 26068 raft_consensus.cc:729] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404b8bbb0ead48be841f47c85c4f79c7, State: Initialized, Role: FOLLOWER
I20260809 12:08:26.586937 26065 consensus_queue.cc:260] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } }
I20260809 12:08:26.587330 26068 consensus_queue.cc:260] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } }
I20260809 12:08:26.588805 25846 heartbeater.cc:503] Master 127.0.147.126:38103 was elected leader, sending a full tablet report...
I20260809 12:08:26.588886 26065 ts_tablet_manager.cc:1428] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260809 12:08:26.589443 26068 ts_tablet_manager.cc:1428] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260809 12:08:26.609390 25661 tablet.cc:2369] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:08:26.642684 26073 raft_consensus.cc:482] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:08:26.643105 26073 raft_consensus.cc:504] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } }
I20260809 12:08:26.644748 26073 leader_election.cc:288] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 970fae78fc5e4191966c340e18f97663 (127.0.147.65:40443), 9edddbbadbad465b8925d15bb4f5798f (127.0.147.67:46883)
W20260809 12:08:26.647022 25848 tablet.cc:2369] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:08:26.652328 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.645283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:39423 (local address 127.0.147.65:40443)
0809 12:08:26.645472 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.645476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.646157 (+ 681us) server_negotiation.cc:407] Connection header received
0809 12:08:26.646506 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.646512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.646597 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.646659 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:26.647885 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.648649 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.650495 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.651283 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.651406 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.651515 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.651918 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.652072 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:08:26.652525 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.645192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40443 (local address 127.0.147.66:39423)
0809 12:08:26.646055 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:26.646077 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:26.646295 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:26.646902 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:26.646906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:26.646946 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:26.647709 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.647727 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.648784 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.648792 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.650329 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.650341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.651555 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.651563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.651757 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.651782 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:08:26.651861 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 12:08:26.651944 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"spinlock_wait_cycles":23808,"thread_start_us":460,"threads_started":1}
I20260809 12:08:26.652594 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.646105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:46883 (local address 127.0.147.66:39955)
0809 12:08:26.646712 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:26.646726 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:26.646870 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:26.647196 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:26.647201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:26.647235 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:26.647774 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.647786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.648893 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.648897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.650356 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:26.650368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.651651 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.651655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:26.651832 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.651891 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 12:08:26.651988 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 12:08:26.652118 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":46,"thread_start_us":284,"threads_started":1}
I20260809 12:08:26.652859 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.646394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:39955 (local address 127.0.147.67:46883)
0809 12:08:26.646614 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.646620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.646821 (+ 201us) server_negotiation.cc:407] Connection header received
0809 12:08:26.647020 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.647029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.647088 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.647182 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:26.647957 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.648752 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.650544 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.651369 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.651559 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.651636 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.652095 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.652220 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:08:26.653611 25598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab67988b9e640adaadfb2d786ac35ff" candidate_uuid: "404b8bbb0ead48be841f47c85c4f79c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "970fae78fc5e4191966c340e18f97663" is_pre_election: true
I20260809 12:08:26.653939 25969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab67988b9e640adaadfb2d786ac35ff" candidate_uuid: "404b8bbb0ead48be841f47c85c4f79c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edddbbadbad465b8925d15bb4f5798f" is_pre_election: true
I20260809 12:08:26.654234 25598 raft_consensus.cc:2454] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 404b8bbb0ead48be841f47c85c4f79c7 in term 0.
I20260809 12:08:26.654524 25969 raft_consensus.cc:2454] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 404b8bbb0ead48be841f47c85c4f79c7 in term 0.
I20260809 12:08:26.655037 25695 leader_election.cc:302] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 404b8bbb0ead48be841f47c85c4f79c7, 970fae78fc5e4191966c340e18f97663; no voters:
I20260809 12:08:26.655392 26073 raft_consensus.cc:2781] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:08:26.655566 26073 raft_consensus.cc:482] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:08:26.655678 26073 raft_consensus.cc:3037] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:26.657924 26073 raft_consensus.cc:504] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } }
I20260809 12:08:26.658546 26073 leader_election.cc:288] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [CANDIDATE]: Term 1 election: Requested vote from peers 970fae78fc5e4191966c340e18f97663 (127.0.147.65:40443), 9edddbbadbad465b8925d15bb4f5798f (127.0.147.67:46883)
I20260809 12:08:26.659004 25598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab67988b9e640adaadfb2d786ac35ff" candidate_uuid: "404b8bbb0ead48be841f47c85c4f79c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "970fae78fc5e4191966c340e18f97663"
I20260809 12:08:26.659108 25969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab67988b9e640adaadfb2d786ac35ff" candidate_uuid: "404b8bbb0ead48be841f47c85c4f79c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edddbbadbad465b8925d15bb4f5798f"
I20260809 12:08:26.659214 25598 raft_consensus.cc:3037] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:26.659359 25969 raft_consensus.cc:3037] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:26.661491 25598 raft_consensus.cc:2454] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 404b8bbb0ead48be841f47c85c4f79c7 in term 1.
I20260809 12:08:26.661716 25969 raft_consensus.cc:2454] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 404b8bbb0ead48be841f47c85c4f79c7 in term 1.
I20260809 12:08:26.661931 25695 leader_election.cc:302] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 404b8bbb0ead48be841f47c85c4f79c7, 970fae78fc5e4191966c340e18f97663; no voters:
I20260809 12:08:26.662237 26073 raft_consensus.cc:2781] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:26.662693 26073 raft_consensus.cc:686] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 1 LEADER]: Becoming Leader. State: Replica: 404b8bbb0ead48be841f47c85c4f79c7, State: Running, Role: LEADER
I20260809 12:08:26.663148 26073 consensus_queue.cc:237] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } }
I20260809 12:08:26.667788 25405 catalog_manager.cc:5495] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 404b8bbb0ead48be841f47c85c4f79c7 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "404b8bbb0ead48be841f47c85c4f79c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "404b8bbb0ead48be841f47c85c4f79c7" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 46303 } health_report { overall_health: HEALTHY } } }
I20260809 12:08:26.683966 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.678798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56238 (local address 127.0.147.66:46303)
0809 12:08:26.678954 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.678958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.678974 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:08:26.679050 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.679053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.679123 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.679239 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:08:26.679636 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.680474 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.682074 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.682472 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.682574 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.683005 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:08:26.683457 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:08:26.683650 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.683701 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.683761 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 12:08:26.691736 25969 raft_consensus.cc:1261] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [term 1 FOLLOWER]: Refusing update from remote peer 404b8bbb0ead48be841f47c85c4f79c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:26.691742 25598 raft_consensus.cc:1261] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [term 1 FOLLOWER]: Refusing update from remote peer 404b8bbb0ead48be841f47c85c4f79c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:26.692696 26082 consensus_queue.cc:1035] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "970fae78fc5e4191966c340e18f97663" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:26.693082 26073 consensus_queue.cc:1035] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9edddbbadbad465b8925d15bb4f5798f" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 46883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:08:26.696739 26041 tablet.cc:2369] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:08:26.725805 25405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:26.726178 25405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:26.730055 25405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd9c7ca213c14676a640dc472505981b: Sending DeleteTablet for 3 replicas of tablet dab67988b9e640adaadfb2d786ac35ff
I20260809 12:08:26.731201 25565 tablet_service.cc:1514] Processing DeleteTablet for tablet dab67988b9e640adaadfb2d786ac35ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:26 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:51882
I20260809 12:08:26.731320 25947 tablet_service.cc:1514] Processing DeleteTablet for tablet dab67988b9e640adaadfb2d786ac35ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:26 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:43048
I20260809 12:08:26.731518 25756 tablet_service.cc:1514] Processing DeleteTablet for tablet dab67988b9e640adaadfb2d786ac35ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:26 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:56234
I20260809 12:08:26.732537 26100 tablet_replica.cc:330] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: stopping tablet replica
I20260809 12:08:26.733029 26099 tablet_replica.cc:330] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: stopping tablet replica
I20260809 12:08:26.733156 26100 raft_consensus.cc:2229] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:08:26.733430 26099 raft_consensus.cc:2229] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:26.733747 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 25483
I20260809 12:08:26.733999 26099 raft_consensus.cc:2258] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:26.734159 26100 raft_consensus.cc:2258] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:26.734743 26098 tablet_replica.cc:330] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: stopping tablet replica
I20260809 12:08:26.735184 26098 raft_consensus.cc:2229] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:26.735721 26098 raft_consensus.cc:2258] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:26.738300 26100 ts_tablet_manager.cc:1905] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:26.740296 26099 ts_tablet_manager.cc:1905] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:26.741184 26098 ts_tablet_manager.cc:1905] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 12:08:26 dist-test-slave-sb2z krb5kdc[25348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:08:26.747977 26099 ts_tablet_manager.cc:1918] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:26.748162 26099 log.cc:1196] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/dab67988b9e640adaadfb2d786ac35ff
I20260809 12:08:26.748749 26099 ts_tablet_manager.cc:1939] T dab67988b9e640adaadfb2d786ac35ff P 9edddbbadbad465b8925d15bb4f5798f: Deleting consensus metadata
I20260809 12:08:26.749431 26098 ts_tablet_manager.cc:1918] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:26.749619 26098 log.cc:1196] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/dab67988b9e640adaadfb2d786ac35ff
I20260809 12:08:26.750190 26098 ts_tablet_manager.cc:1939] T dab67988b9e640adaadfb2d786ac35ff P 970fae78fc5e4191966c340e18f97663: Deleting consensus metadata
I20260809 12:08:26.750689 25387 catalog_manager.cc:4841] TS 9edddbbadbad465b8925d15bb4f5798f (127.0.147.67:46883): tablet dab67988b9e640adaadfb2d786ac35ff (table test-table [id=27d6ce44c5c54a2f812017dbd3b13c26]) successfully deleted
I20260809 12:08:26.751982 25387 catalog_manager.cc:4841] TS 970fae78fc5e4191966c340e18f97663 (127.0.147.65:40443): tablet dab67988b9e640adaadfb2d786ac35ff (table test-table [id=27d6ce44c5c54a2f812017dbd3b13c26]) successfully deleted
I20260809 12:08:26.756525 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.735849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51894 (local address 127.0.147.65:40443)
0809 12:08:26.736608 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.736614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.737032 (+ 418us) server_negotiation.cc:407] Connection header received
0809 12:08:26.737329 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.737336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.737412 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.737468 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:26.738942 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.739865 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.742027 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.742679 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.742780 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.749142 (+ 6362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:26.749159 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:26.749179 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:26.749231 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:26.753645 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:26.754385 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:26.754394 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:26.754397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:26.754536 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:26.754930 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:26.754935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:26.754939 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:26.755275 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:26.755639 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.755922 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.756140 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617}
I20260809 12:08:26.757242 25653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:26.757261 26100 ts_tablet_manager.cc:1918] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:26.757458 26100 log.cc:1196] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/dab67988b9e640adaadfb2d786ac35ff
I20260809 12:08:26.758064 26100 ts_tablet_manager.cc:1939] T dab67988b9e640adaadfb2d786ac35ff P 404b8bbb0ead48be841f47c85c4f79c7: Deleting consensus metadata
I20260809 12:08:26.759852 25390 catalog_manager.cc:4841] TS 404b8bbb0ead48be841f47c85c4f79c7 (127.0.147.66:46303): tablet dab67988b9e640adaadfb2d786ac35ff (table test-table [id=27d6ce44c5c54a2f812017dbd3b13c26]) successfully deleted
I20260809 12:08:26.859946 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 25483
I20260809 12:08:26.892632 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 25667
Aug 09 12:08:26 dist-test-slave-sb2z krb5kdc[25348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:08:26.913389 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:26.893525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.0.147.66:46303)
0809 12:08:26.893739 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:26.893743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:26.893766 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:08:26.894007 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:26.894013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:26.894115 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:26.894224 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:26.895955 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.897300 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.899245 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:26.900264 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:26.900401 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:26.905942 (+ 5541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:26.905959 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:26.905978 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:26.906028 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:26.910159 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:26.911040 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:26.911053 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:26.911057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:26.911260 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:26.911702 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:26.911707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:26.911710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:26.912162 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:26.912481 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:26.912796 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 12:08:26.913021 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:08:26.914136 25837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:27.030206 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 25667
I20260809 12:08:27.059671 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 25857
Aug 09 12:08:27 dist-test-slave-sb2z krb5kdc[25348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:08:27.075762 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:27.060481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.0.147.67:46883)
0809 12:08:27.060685 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:27.060692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:27.060779 (+ 87us) server_negotiation.cc:407] Connection header received
0809 12:08:27.061019 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:27.061023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:27.061094 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:27.061204 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:27.062574 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:27.063321 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:27.064942 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:27.065360 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:27.065502 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:27.070079 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:27.070093 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:27.070109 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:27.070146 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:27.072889 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:27.073537 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:27.073545 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:27.073547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:27.073697 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:27.074085 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:27.074092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:27.074100 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:27.074456 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:27.074771 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:27.075120 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 12:08:27.075370 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 12:08:27.076388 26032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:27.167814 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 25857
I20260809 12:08:27.189867 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 25364
I20260809 12:08:27.190984 25443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:27.303174 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 25364
2026-08-09T12:08:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4743 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T12:08:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:27Z Disabled control of system clock
I20260809 12:08:27.419083 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:40339
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:40339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:08:27.758796 26115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:27.759083 26115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:27.759167 26115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:27.766484 26115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:27.766592 26115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:27.766630 26115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:27.766690 26115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:27.766737 26115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:27.775919 26115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:40339
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:40339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:27.777137 26115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:27.778888 26115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:27.791074 26121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:27.791028 26120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:27.791116 26124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:27.791734 26115 server_base.cc:1034] running on GCE node
I20260809 12:08:27.792419 26115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:27.793672 26115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:27.794915 26115 hybrid_clock.cc:648] HybridClock initialized: now 1786277307794842 us; error 51 us; skew 500 ppm
I20260809 12:08:27.795322 26115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:27.802438 26115 webserver.cc:460] Webserver started at http://127.0.147.126:42799/ using document root <none> and password file <none>
I20260809 12:08:27.803122 26115 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:27.803241 26115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:27.803658 26115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:27.806571 26115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "faaf696a1e784d69b948d1a7f256f213"
format_stamp: "Formatted at 2026-08-09 12:08:27 on dist-test-slave-sb2z"
server_key: "d3b5c21481a5b8f71c86470ae6366759"
server_key_iv: "c3559fd955a376eb11e24e7149a3049f"
server_key_version: "encryptionkey@0"
I20260809 12:08:27.807369 26115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "faaf696a1e784d69b948d1a7f256f213"
format_stamp: "Formatted at 2026-08-09 12:08:27 on dist-test-slave-sb2z"
server_key: "d3b5c21481a5b8f71c86470ae6366759"
server_key_iv: "c3559fd955a376eb11e24e7149a3049f"
server_key_version: "encryptionkey@0"
I20260809 12:08:27.812354 26115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 12:08:27.815876 26134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:27.817114 26115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:08:27.817302 26115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "faaf696a1e784d69b948d1a7f256f213"
format_stamp: "Formatted at 2026-08-09 12:08:27 on dist-test-slave-sb2z"
server_key: "d3b5c21481a5b8f71c86470ae6366759"
server_key_iv: "c3559fd955a376eb11e24e7149a3049f"
server_key_version: "encryptionkey@0"
I20260809 12:08:27.817510 26115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:27.839910 26115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:27.840725 26115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:27.841053 26115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:27.859591 26115 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:40339
I20260809 12:08:27.859634 26205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:40339 every 8 connection(s)
I20260809 12:08:27.860950 26115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:27.864595 26208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:27.867240 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 26115
I20260809 12:08:27.867416 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:27.867925 589 external_mini_cluster.cc:1428] Setting key f99fe83eab8f92dd36ac6d20cc1c4d73
I20260809 12:08:27.875926 26208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213: Bootstrap starting.
I20260809 12:08:27.878942 26208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:27.880115 26208 log.cc:824] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:27.881389 26211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:27.869801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.0.147.126:40339)
0809 12:08:27.870818 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:27.870828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:27.870869 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:08:27.872575 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:27.872597 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:27.873054 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:27.873445 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:27.874089 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:27.875331 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:27.876865 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:27.877558 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:27.877811 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:27.878970 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:27.878978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:27.878992 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:27.879690 (+ 698us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:27.879757 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:27.879777 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:27.879914 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:27.880171 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 12:08:27.880453 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":361,"threads_started":1}
I20260809 12:08:27.883291 26208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213: No bootstrap required, opened a new log
I20260809 12:08:27.890753 26208 raft_consensus.cc:348] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaf696a1e784d69b948d1a7f256f213" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40339 } }
I20260809 12:08:27.891237 26208 raft_consensus.cc:374] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:27.891350 26208 raft_consensus.cc:729] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faaf696a1e784d69b948d1a7f256f213, State: Initialized, Role: FOLLOWER
I20260809 12:08:27.891968 26208 consensus_queue.cc:260] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaf696a1e784d69b948d1a7f256f213" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40339 } }
I20260809 12:08:27.892217 26208 raft_consensus.cc:388] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:27.892372 26208 raft_consensus.cc:482] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:27.892542 26208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:27.894654 26208 raft_consensus.cc:504] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaf696a1e784d69b948d1a7f256f213" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40339 } }
I20260809 12:08:27.895181 26208 leader_election.cc:302] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faaf696a1e784d69b948d1a7f256f213; no voters:
I20260809 12:08:27.895785 26208 leader_election.cc:288] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:27.895937 26215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:27.896665 26215 raft_consensus.cc:686] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [term 1 LEADER]: Becoming Leader. State: Replica: faaf696a1e784d69b948d1a7f256f213, State: Running, Role: LEADER
I20260809 12:08:27.897217 26215 consensus_queue.cc:237] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaf696a1e784d69b948d1a7f256f213" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40339 } }
I20260809 12:08:27.897862 26208 sys_catalog.cc:564] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:27.900100 26218 sys_catalog.cc:455] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faaf696a1e784d69b948d1a7f256f213. Latest consensus state: current_term: 1 leader_uuid: "faaf696a1e784d69b948d1a7f256f213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaf696a1e784d69b948d1a7f256f213" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40339 } } }
I20260809 12:08:27.900321 26216 sys_catalog.cc:455] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faaf696a1e784d69b948d1a7f256f213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaf696a1e784d69b948d1a7f256f213" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 40339 } } }
I20260809 12:08:27.900621 26218 sys_catalog.cc:458] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:27.900650 26216 sys_catalog.cc:458] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:27.902498 26224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:27.907691 26224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:27.916996 26224 catalog_manager.cc:1347] Generated new cluster ID: 01f0144679f543d09a912b8222f40bcf
I20260809 12:08:27.917138 26224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:27.943791 26224 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:27.945228 26224 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:27.971205 26224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213: Generated new TSK 0
I20260809 12:08:27.972123 26224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:08:27.978722 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40339
--builtin_ntp_servers=127.0.147.84:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:08:28.343611 26239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:28.343947 26239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:28.344123 26239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:28.351476 26239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:28.351624 26239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:28.351755 26239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:08:28.360721 26239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:28.361920 26239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:28.363636 26239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:28.376910 26246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:28.376929 26256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:28.376959 26253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:28.378189 26239 server_base.cc:1034] running on GCE node
I20260809 12:08:28.378861 26239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:28.379824 26239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:28.381052 26239 hybrid_clock.cc:648] HybridClock initialized: now 1786277308381014 us; error 52 us; skew 500 ppm
I20260809 12:08:28.381371 26239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:28.383603 26239 webserver.cc:460] Webserver started at http://127.0.147.65:35435/ using document root <none> and password file <none>
I20260809 12:08:28.384274 26239 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:28.384411 26239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:28.384742 26239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:28.387629 26239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "a84fba94eb7a40aa88cdc33a34ac880e"
format_stamp: "Formatted at 2026-08-09 12:08:28 on dist-test-slave-sb2z"
server_key: "05989214392a7c3b0967fdea7de7dbb8"
server_key_iv: "d268d3998bb88a04fe2a751235271c6b"
server_key_version: "encryptionkey@0"
I20260809 12:08:28.388413 26239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "a84fba94eb7a40aa88cdc33a34ac880e"
format_stamp: "Formatted at 2026-08-09 12:08:28 on dist-test-slave-sb2z"
server_key: "05989214392a7c3b0967fdea7de7dbb8"
server_key_iv: "d268d3998bb88a04fe2a751235271c6b"
server_key_version: "encryptionkey@0"
I20260809 12:08:28.393219 26239 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 12:08:28.396548 26263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:28.397967 26239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:08:28.398103 26239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "a84fba94eb7a40aa88cdc33a34ac880e"
format_stamp: "Formatted at 2026-08-09 12:08:28 on dist-test-slave-sb2z"
server_key: "05989214392a7c3b0967fdea7de7dbb8"
server_key_iv: "d268d3998bb88a04fe2a751235271c6b"
server_key_version: "encryptionkey@0"
I20260809 12:08:28.398319 26239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:28.412161 26239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:28.412945 26239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:28.413257 26239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:28.414202 26239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:28.415671 26239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:28.415782 26239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:28.415918 26239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:28.416028 26239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:28.448118 26239 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:39943
I20260809 12:08:28.448138 26420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:39943 every 8 connection(s)
I20260809 12:08:28.449452 26239 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:08:28.458477 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 26239
I20260809 12:08:28.458670 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:08:28.459143 589 external_mini_cluster.cc:1428] Setting key 2fb2b83e13005611234dd7c057cdf192
I20260809 12:08:28.461925 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40339
--builtin_ntp_servers=127.0.147.84:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:08:28.462229 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:28.452087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:33489 (local address 127.0.147.126:40339)
0809 12:08:28.452520 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:28.452525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:28.453727 (+ 1202us) server_negotiation.cc:407] Connection header received
0809 12:08:28.455541 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:28.455548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:28.455643 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:28.455768 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:28.456813 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:28.457535 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:28.459181 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:28.459705 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:28.459840 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:28.461379 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:28.461389 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:28.461392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:28.461408 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:28.461450 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:28.461463 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:28.461565 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:28.461789 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 12:08:28.461884 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":218,"threads_started":1}
I20260809 12:08:28.463176 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:28.452681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40339 (local address 127.0.147.65:33489)
0809 12:08:28.453572 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:28.453613 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:28.455262 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:28.455998 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:28.456007 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:28.456093 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:28.456643 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:28.456660 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:28.457698 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:28.457702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:28.459027 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:28.459040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:28.459940 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:28.459948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:28.460248 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:28.461062 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:08:28.461065 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:28.461120 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:08:28.461190 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:28.461575 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:28.461588 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:28.461598 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:08:28.461779 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 12:08:28.462155 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":297,"threads_started":1}
I20260809 12:08:28.464769 26421 heartbeater.cc:347] Connected to a master server at 127.0.147.126:40339
I20260809 12:08:28.465067 26421 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:28.465782 26421 heartbeater.cc:511] Master 127.0.147.126:40339 requested a full tablet report, sending...
I20260809 12:08:28.467862 26152 ts_manager.cc:194] Registered new tserver with Master: a84fba94eb7a40aa88cdc33a34ac880e (127.0.147.65:39943)
I20260809 12:08:28.469350 26152 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.65:33489
W20260809 12:08:28.806977 26428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:28.807238 26428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:28.807380 26428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:28.814641 26428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:28.814765 26428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:28.814932 26428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:08:28.824077 26428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:28.825280 26428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:28.827101 26428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:28.840376 26434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:28.840428 26437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:28.840391 26435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:28.841696 26428 server_base.cc:1034] running on GCE node
I20260809 12:08:28.842298 26428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:28.843221 26428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:28.844431 26428 hybrid_clock.cc:648] HybridClock initialized: now 1786277308844358 us; error 58 us; skew 500 ppm
I20260809 12:08:28.844758 26428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:28.847057 26428 webserver.cc:460] Webserver started at http://127.0.147.66:42681/ using document root <none> and password file <none>
I20260809 12:08:28.847853 26428 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:28.847980 26428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:28.848321 26428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:28.851085 26428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "677871a664f542ad9a2bddfa23352687"
format_stamp: "Formatted at 2026-08-09 12:08:28 on dist-test-slave-sb2z"
server_key: "ee18044766793e25d554795da9c86f00"
server_key_iv: "d9e70c58de28efae47785cafcbc47b29"
server_key_version: "encryptionkey@0"
I20260809 12:08:28.851930 26428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "677871a664f542ad9a2bddfa23352687"
format_stamp: "Formatted at 2026-08-09 12:08:28 on dist-test-slave-sb2z"
server_key: "ee18044766793e25d554795da9c86f00"
server_key_iv: "d9e70c58de28efae47785cafcbc47b29"
server_key_version: "encryptionkey@0"
I20260809 12:08:28.856994 26428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:08:28.860669 26448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:28.862152 26428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:08:28.862305 26428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "677871a664f542ad9a2bddfa23352687"
format_stamp: "Formatted at 2026-08-09 12:08:28 on dist-test-slave-sb2z"
server_key: "ee18044766793e25d554795da9c86f00"
server_key_iv: "d9e70c58de28efae47785cafcbc47b29"
server_key_version: "encryptionkey@0"
I20260809 12:08:28.862545 26428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:28.897183 26428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:28.898058 26428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:28.898376 26428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:28.899403 26428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:28.900837 26428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:28.900946 26428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:28.901029 26428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:28.901079 26428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:28.935735 26428 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:41771
I20260809 12:08:28.935824 26614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:41771 every 8 connection(s)
I20260809 12:08:28.937036 26428 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:08:28.945909 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 26428
I20260809 12:08:28.946079 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:08:28.946718 589 external_mini_cluster.cc:1428] Setting key c4322e6d4c53140fff7e537783e2452a
I20260809 12:08:28.949527 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:28.939605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:48107 (local address 127.0.147.126:40339)
0809 12:08:28.939779 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:28.939783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:28.941121 (+ 1338us) server_negotiation.cc:407] Connection header received
0809 12:08:28.942828 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:28.942832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:28.942901 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:28.942998 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:28.944067 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:28.944781 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:28.946405 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:28.947033 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:28.947169 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:28.948661 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:28.948667 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:28.948672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:28.948693 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:28.948734 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:28.948754 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:28.948822 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:28.949118 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 12:08:28.949210 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:08:28.949983 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40339
--builtin_ntp_servers=127.0.147.84:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:08:28.950507 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:28.940074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40339 (local address 127.0.147.66:48107)
0809 12:08:28.940953 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:28.940988 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:28.942593 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:28.943194 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:28.943201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:28.943279 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:28.943900 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:28.943917 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:28.944902 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:28.944906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:28.946248 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:28.946261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:28.947230 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:28.947235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:28.947531 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:28.948298 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:08:28.948302 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:28.948349 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:08:28.948440 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:28.948856 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:28.948868 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:28.948876 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:08:28.949056 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 12:08:28.949319 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":270,"threads_started":1}
I20260809 12:08:28.951889 26615 heartbeater.cc:347] Connected to a master server at 127.0.147.126:40339
I20260809 12:08:28.952191 26615 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:28.952873 26615 heartbeater.cc:511] Master 127.0.147.126:40339 requested a full tablet report, sending...
I20260809 12:08:28.954198 26152 ts_manager.cc:194] Registered new tserver with Master: 677871a664f542ad9a2bddfa23352687 (127.0.147.66:41771)
I20260809 12:08:28.954999 26152 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.66:48107
W20260809 12:08:29.297058 26618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:29.297379 26618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:29.297542 26618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:29.305117 26618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:29.305245 26618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:29.305428 26618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:08:29.314256 26618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:40339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:29.315538 26618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:29.317360 26618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:29.332410 26627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:29.332458 26629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:29.332471 26633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:29.333799 26618 server_base.cc:1034] running on GCE node
I20260809 12:08:29.334491 26618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:29.335404 26618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:29.336642 26618 hybrid_clock.cc:648] HybridClock initialized: now 1786277309336614 us; error 63 us; skew 500 ppm
I20260809 12:08:29.336959 26618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:29.339210 26618 webserver.cc:460] Webserver started at http://127.0.147.67:35555/ using document root <none> and password file <none>
I20260809 12:08:29.339960 26618 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:29.340096 26618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:29.340428 26618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:29.343187 26618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "b025089f6b444f24bab9c179513111b0"
format_stamp: "Formatted at 2026-08-09 12:08:29 on dist-test-slave-sb2z"
server_key: "cedabf6c651dc529b7d356063844fc58"
server_key_iv: "81874d3bdca2250487fe4de87c43ac8f"
server_key_version: "encryptionkey@0"
I20260809 12:08:29.344004 26618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "b025089f6b444f24bab9c179513111b0"
format_stamp: "Formatted at 2026-08-09 12:08:29 on dist-test-slave-sb2z"
server_key: "cedabf6c651dc529b7d356063844fc58"
server_key_iv: "81874d3bdca2250487fe4de87c43ac8f"
server_key_version: "encryptionkey@0"
I20260809 12:08:29.348927 26618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:08:29.352330 26641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:29.353658 26618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:08:29.353832 26618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "b025089f6b444f24bab9c179513111b0"
format_stamp: "Formatted at 2026-08-09 12:08:29 on dist-test-slave-sb2z"
server_key: "cedabf6c651dc529b7d356063844fc58"
server_key_iv: "81874d3bdca2250487fe4de87c43ac8f"
server_key_version: "encryptionkey@0"
I20260809 12:08:29.354048 26618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:29.377707 26618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:29.378520 26618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:29.378839 26618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:29.379915 26618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:29.381381 26618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:29.381510 26618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:29.381641 26618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:29.381731 26618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:29.427875 26618 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:43953
I20260809 12:08:29.427969 26795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:43953 every 8 connection(s)
I20260809 12:08:29.429203 26618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:08:29.434765 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 26618
I20260809 12:08:29.434950 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:08:29.435438 589 external_mini_cluster.cc:1428] Setting key e4f095464f37ef039df97c2c126ed672
I20260809 12:08:29.442315 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.431791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:50323 (local address 127.0.147.126:40339)
0809 12:08:29.432005 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.432010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.433414 (+ 1404us) server_negotiation.cc:407] Connection header received
0809 12:08:29.435638 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.435643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.435732 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.435856 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:29.437000 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.437783 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.439410 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.439889 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.440108 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.441484 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:29.441490 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:29.441495 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:29.441520 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:29.441552 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:29.441565 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:29.441663 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.441880 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.441990 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:08:29.443339 26799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.432376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:40339 (local address 127.0.147.67:50323)
0809 12:08:29.433259 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.433294 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:29.435227 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:29.436034 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:29.436042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:29.436133 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:29.436827 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.436845 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.437949 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.437954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.439261 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.439273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.440196 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.440203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.440453 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.441162 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:08:29.441164 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:29.441203 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:08:29.441286 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:29.441662 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:29.441675 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:29.441684 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 12:08:29.441870 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 12:08:29.442258 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":275,"threads_started":1}
I20260809 12:08:29.444784 26796 heartbeater.cc:347] Connected to a master server at 127.0.147.126:40339
I20260809 12:08:29.445142 26796 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:29.445923 26796 heartbeater.cc:511] Master 127.0.147.126:40339 requested a full tablet report, sending...
I20260809 12:08:29.447367 26152 ts_manager.cc:194] Registered new tserver with Master: b025089f6b444f24bab9c179513111b0 (127.0.147.67:43953)
I20260809 12:08:29.448204 26152 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.67:50323
I20260809 12:08:29.450743 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:08:29.461601 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.455151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52252 (local address 127.0.147.126:40339)
0809 12:08:29.455347 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.455352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.455782 (+ 430us) server_negotiation.cc:407] Connection header received
0809 12:08:29.455912 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.455915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.455995 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.456092 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:29.456561 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.457390 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.459178 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.459666 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.459767 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.460794 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:29.460798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:29.460802 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:29.460818 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:29.460850 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:29.460862 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:29.460945 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.461264 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.461340 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:08:29.465541 26152 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 12:08:29.472517 26421 heartbeater.cc:503] Master 127.0.147.126:40339 was elected leader, sending a full tablet report...
W20260809 12:08:29.472589 26152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:08:29.494047 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.484804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.0.147.67:43953)
0809 12:08:29.485524 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.485535 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.485979 (+ 444us) server_negotiation.cc:407] Connection header received
0809 12:08:29.486274 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.486283 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.486532 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.486748 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:29.487852 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.489498 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.491744 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.492791 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.492966 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.493244 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.493473 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.493646 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":343,"threads_started":1}
I20260809 12:08:29.494132 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.484781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43953 (local address 127.0.0.1:57650)
0809 12:08:29.485834 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.485871 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:29.486064 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:29.487147 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:29.487163 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:29.487220 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:29.487630 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.487649 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.489649 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.489656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.491473 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.491487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.493015 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.493022 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.493218 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.493275 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 12:08:29.493411 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 12:08:29.493625 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":4,"thread_start_us":709,"threads_started":1}
I20260809 12:08:29.497080 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.486823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.0.1:37228)
0809 12:08:29.487376 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.487396 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:29.487546 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:29.488893 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:29.488898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:29.488932 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:29.489304 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.489315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.491380 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.491387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.494172 (+ 2785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.494193 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.496261 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.496269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.496493 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.496549 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 12:08:29.496631 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 12:08:29.496766 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":836,"threads_started":1}
I20260809 12:08:29.497645 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.486179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:41771 (local address 127.0.0.1:46328)
0809 12:08:29.486755 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.486772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:29.486880 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:29.489313 (+ 2433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:29.489317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:29.489343 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:29.489827 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.489838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.491856 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.491864 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.494696 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.494711 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.496567 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.496574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.496748 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.496787 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:08:29.496873 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 12:08:29.497010 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":21,"thread_start_us":308,"threads_started":1}
I20260809 12:08:29.497715 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.486498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46328 (local address 127.0.147.66:41771)
0809 12:08:29.488789 (+ 2291us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.488797 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.488822 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:08:29.488915 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.488921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.489107 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.489300 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:29.490017 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.491666 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.494913 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.496353 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.496522 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.496755 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.497185 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.497348 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2150,"thread_start_us":2039,"threads_started":1}
I20260809 12:08:29.497740 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.487423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37228 (local address 127.0.147.65:39943)
0809 12:08:29.488280 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.488291 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.488326 (+ 35us) server_negotiation.cc:407] Connection header received
0809 12:08:29.488451 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.488460 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.488698 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.488906 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:29.489548 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.491063 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.494448 (+ 3385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.496019 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.496883 (+ 864us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.497148 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.497213 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.497366 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":517,"threads_started":1}
I20260809 12:08:29.498709 26710 tablet_service.cc:1467] Processing CreateTablet for tablet d49c67ce2bd047dd9cb3497eb6c2ff63 (DEFAULT_TABLE table=test-table [id=1dc5bbe27c1f4fa9a7d4ea54bc82e7ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:29.500880 26710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d49c67ce2bd047dd9cb3497eb6c2ff63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:29.501107 26330 tablet_service.cc:1467] Processing CreateTablet for tablet d49c67ce2bd047dd9cb3497eb6c2ff63 (DEFAULT_TABLE table=test-table [id=1dc5bbe27c1f4fa9a7d4ea54bc82e7ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:29.501583 26519 tablet_service.cc:1467] Processing CreateTablet for tablet d49c67ce2bd047dd9cb3497eb6c2ff63 (DEFAULT_TABLE table=test-table [id=1dc5bbe27c1f4fa9a7d4ea54bc82e7ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:29.502444 26330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d49c67ce2bd047dd9cb3497eb6c2ff63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:29.503042 26519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d49c67ce2bd047dd9cb3497eb6c2ff63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:29.514287 26823 tablet_bootstrap.cc:492] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e: Bootstrap starting.
I20260809 12:08:29.516759 26825 tablet_bootstrap.cc:492] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: Bootstrap starting.
I20260809 12:08:29.518862 26823 tablet_bootstrap.cc:654] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:29.520226 26823 log.cc:824] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:29.520265 26827 tablet_bootstrap.cc:492] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: Bootstrap starting.
I20260809 12:08:29.520373 26825 tablet_bootstrap.cc:654] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:29.521833 26825 log.cc:824] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:29.524161 26827 tablet_bootstrap.cc:654] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:29.525005 26825 tablet_bootstrap.cc:492] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: No bootstrap required, opened a new log
I20260809 12:08:29.525281 26825 ts_tablet_manager.cc:1397] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 12:08:29.525375 26823 tablet_bootstrap.cc:492] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e: No bootstrap required, opened a new log
I20260809 12:08:29.525596 26827 log.cc:824] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:29.525784 26823 ts_tablet_manager.cc:1397] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260809 12:08:29.530032 26827 tablet_bootstrap.cc:492] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: No bootstrap required, opened a new log
I20260809 12:08:29.530301 26827 ts_tablet_manager.cc:1397] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 12:08:29.533293 26825 raft_consensus.cc:348] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } }
I20260809 12:08:29.533674 26825 raft_consensus.cc:374] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:29.533797 26825 raft_consensus.cc:729] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b025089f6b444f24bab9c179513111b0, State: Initialized, Role: FOLLOWER
I20260809 12:08:29.534430 26825 consensus_queue.cc:260] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } }
I20260809 12:08:29.535517 26796 heartbeater.cc:503] Master 127.0.147.126:40339 was elected leader, sending a full tablet report...
I20260809 12:08:29.536257 26825 ts_tablet_manager.cc:1428] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260809 12:08:29.537374 26823 raft_consensus.cc:348] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } }
I20260809 12:08:29.537815 26823 raft_consensus.cc:374] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:29.537926 26823 raft_consensus.cc:729] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84fba94eb7a40aa88cdc33a34ac880e, State: Initialized, Role: FOLLOWER
I20260809 12:08:29.538515 26823 consensus_queue.cc:260] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } }
I20260809 12:08:29.540283 26823 ts_tablet_manager.cc:1428] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260809 12:08:29.541002 26827 raft_consensus.cc:348] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } }
I20260809 12:08:29.541451 26827 raft_consensus.cc:374] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:29.541561 26827 raft_consensus.cc:729] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 677871a664f542ad9a2bddfa23352687, State: Initialized, Role: FOLLOWER
I20260809 12:08:29.542198 26827 consensus_queue.cc:260] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } }
I20260809 12:08:29.543310 26615 heartbeater.cc:503] Master 127.0.147.126:40339 was elected leader, sending a full tablet report...
I20260809 12:08:29.543818 26827 ts_tablet_manager.cc:1428] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: Time spent starting tablet: real 0.013s user 0.006s sys 0.010s
I20260809 12:08:29.567747 26833 raft_consensus.cc:482] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:08:29.568123 26833 raft_consensus.cc:504] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } }
I20260809 12:08:29.569767 26833 leader_election.cc:288] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b025089f6b444f24bab9c179513111b0 (127.0.147.67:43953), a84fba94eb7a40aa88cdc33a34ac880e (127.0.147.65:39943)
I20260809 12:08:29.576586 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.570308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:56309 (local address 127.0.147.67:43953)
0809 12:08:29.570509 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.570514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.571086 (+ 572us) server_negotiation.cc:407] Connection header received
0809 12:08:29.571387 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.571393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.571475 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.571572 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:29.572250 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.572982 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.574805 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.575674 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.575857 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.575930 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.576208 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.576334 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:08:29.576653 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.570190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43953 (local address 127.0.147.66:56309)
0809 12:08:29.570974 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.570995 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:29.571145 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:29.571751 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:29.571754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:29.571776 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:29.572103 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.572110 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.573129 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.573135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.574606 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.574618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.575870 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.575876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.576044 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.576068 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:08:29.576178 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 12:08:29.576312 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":408,"threads_started":1}
I20260809 12:08:29.577026 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.571275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:39025 (local address 127.0.147.65:39943)
0809 12:08:29.571493 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.571502 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.571775 (+ 273us) server_negotiation.cc:407] Connection header received
0809 12:08:29.571999 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.572011 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.572072 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.572168 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:29.572688 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.573573 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.575281 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.576106 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.576301 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.576370 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.576637 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.576795 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 12:08:29.577167 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.571057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.147.66:39025)
0809 12:08:29.571664 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.571682 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:29.571835 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:29.572193 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:29.572197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:29.572221 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:29.572533 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.572541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.573714 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.573720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.575099 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:29.575111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.576302 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.576308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:29.576474 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.576495 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 12:08:29.576601 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:08:29.576863 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":290,"threads_started":1}
I20260809 12:08:29.577891 26738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d49c67ce2bd047dd9cb3497eb6c2ff63" candidate_uuid: "677871a664f542ad9a2bddfa23352687" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b025089f6b444f24bab9c179513111b0" is_pre_election: true
I20260809 12:08:29.578234 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d49c67ce2bd047dd9cb3497eb6c2ff63" candidate_uuid: "677871a664f542ad9a2bddfa23352687" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" is_pre_election: true
I20260809 12:08:29.578420 26738 raft_consensus.cc:2454] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677871a664f542ad9a2bddfa23352687 in term 0.
I20260809 12:08:29.578775 26358 raft_consensus.cc:2454] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677871a664f542ad9a2bddfa23352687 in term 0.
I20260809 12:08:29.579116 26449 leader_election.cc:302] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 677871a664f542ad9a2bddfa23352687, b025089f6b444f24bab9c179513111b0; no voters:
I20260809 12:08:29.579566 26833 raft_consensus.cc:2781] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:08:29.579743 26833 raft_consensus.cc:482] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:08:29.579871 26833 raft_consensus.cc:3037] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:29.582124 26833 raft_consensus.cc:504] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } }
I20260809 12:08:29.582696 26833 leader_election.cc:288] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [CANDIDATE]: Term 1 election: Requested vote from peers b025089f6b444f24bab9c179513111b0 (127.0.147.67:43953), a84fba94eb7a40aa88cdc33a34ac880e (127.0.147.65:39943)
I20260809 12:08:29.583204 26738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d49c67ce2bd047dd9cb3497eb6c2ff63" candidate_uuid: "677871a664f542ad9a2bddfa23352687" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b025089f6b444f24bab9c179513111b0"
I20260809 12:08:29.583280 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d49c67ce2bd047dd9cb3497eb6c2ff63" candidate_uuid: "677871a664f542ad9a2bddfa23352687" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a84fba94eb7a40aa88cdc33a34ac880e"
I20260809 12:08:29.583415 26738 raft_consensus.cc:3037] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:29.583556 26358 raft_consensus.cc:3037] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:29.585805 26738 raft_consensus.cc:2454] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677871a664f542ad9a2bddfa23352687 in term 1.
I20260809 12:08:29.585938 26358 raft_consensus.cc:2454] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677871a664f542ad9a2bddfa23352687 in term 1.
I20260809 12:08:29.586236 26449 leader_election.cc:302] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 677871a664f542ad9a2bddfa23352687, b025089f6b444f24bab9c179513111b0; no voters:
I20260809 12:08:29.586524 26833 raft_consensus.cc:2781] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:29.587211 26833 raft_consensus.cc:686] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 1 LEADER]: Becoming Leader. State: Replica: 677871a664f542ad9a2bddfa23352687, State: Running, Role: LEADER
I20260809 12:08:29.587778 26833 consensus_queue.cc:237] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } }
I20260809 12:08:29.592639 26151 catalog_manager.cc:5495] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 reported cstate change: term changed from 0 to 1, leader changed from <none> to 677871a664f542ad9a2bddfa23352687 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "677871a664f542ad9a2bddfa23352687" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "677871a664f542ad9a2bddfa23352687" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 41771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 } health_report { overall_health: UNKNOWN } } }
I20260809 12:08:29.622929 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.617172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.0.147.66:41771)
0809 12:08:29.617345 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.617350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.617446 (+ 96us) server_negotiation.cc:407] Connection header received
0809 12:08:29.617626 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.617630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.617717 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.617840 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:08:29.618236 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.618931 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.620614 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.621084 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.621192 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.621822 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:08:29.622382 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:08:29.622563 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.622654 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.622719 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:08:29.631031 26738 raft_consensus.cc:1261] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [term 1 FOLLOWER]: Refusing update from remote peer 677871a664f542ad9a2bddfa23352687: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:29.631074 26358 raft_consensus.cc:1261] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [term 1 FOLLOWER]: Refusing update from remote peer 677871a664f542ad9a2bddfa23352687: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:29.632030 26837 consensus_queue.cc:1035] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b025089f6b444f24bab9c179513111b0" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:29.632367 26833 consensus_queue.cc:1035] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a84fba94eb7a40aa88cdc33a34ac880e" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 39943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:29.643930 26841 mvcc.cc:205] Tried to move back new op lower bound from 7316591860238651392 to 7316591860075966464. Current Snapshot: MvccSnapshot[applied={T|T < 7316591860238651392}]
I20260809 12:08:29.657403 26152 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:29.657706 26152 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:29.661170 26152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P faaf696a1e784d69b948d1a7f256f213: Sending DeleteTablet for 3 replicas of tablet d49c67ce2bd047dd9cb3497eb6c2ff63
I20260809 12:08:29.662338 26710 tablet_service.cc:1514] Processing DeleteTablet for tablet d49c67ce2bd047dd9cb3497eb6c2ff63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:29 UTC) from {username='slave'} at 127.0.0.1:57650
I20260809 12:08:29.662698 26519 tablet_service.cc:1514] Processing DeleteTablet for tablet d49c67ce2bd047dd9cb3497eb6c2ff63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:29 UTC) from {username='slave'} at 127.0.0.1:46328
I20260809 12:08:29.662884 26330 tablet_service.cc:1514] Processing DeleteTablet for tablet d49c67ce2bd047dd9cb3497eb6c2ff63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:29 UTC) from {username='slave'} at 127.0.0.1:37228
I20260809 12:08:29.664505 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 26239
I20260809 12:08:29.682430 26860 tablet_replica.cc:330] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: stopping tablet replica
I20260809 12:08:29.682914 26860 raft_consensus.cc:2229] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:29.683097 26864 tablet_replica.cc:330] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e: stopping tablet replica
I20260809 12:08:29.683499 26860 raft_consensus.cc:2258] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:29.683595 26864 raft_consensus.cc:2229] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:29.683966 26863 tablet_replica.cc:330] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: stopping tablet replica
I20260809 12:08:29.684096 26864 raft_consensus.cc:2258] T d49c67ce2bd047dd9cb3497eb6c2ff63 P a84fba94eb7a40aa88cdc33a34ac880e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:29.684790 26863 raft_consensus.cc:2229] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:08:29.685521 26863 raft_consensus.cc:2258] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:29.685909 26860 ts_tablet_manager.cc:1905] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:29.686204 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.665969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37242 (local address 127.0.147.65:39943)
0809 12:08:29.666205 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.666215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.674040 (+ 7825us) server_negotiation.cc:407] Connection header received
0809 12:08:29.675108 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.675116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.675247 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.675340 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:29.676922 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.678504 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.681898 (+ 3394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.682588 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.682720 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.684512 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:29.684539 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:29.684556 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:29.685293 (+ 737us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:29.685352 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:29.685380 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:29.685497 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.685738 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.685854 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:08:29.687341 26415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:29.688019 26863 ts_tablet_manager.cc:1905] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:29.694846 26860 ts_tablet_manager.cc:1918] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:29.695073 26860 log.cc:1196] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/d49c67ce2bd047dd9cb3497eb6c2ff63
I20260809 12:08:29.695778 26860 ts_tablet_manager.cc:1939] T d49c67ce2bd047dd9cb3497eb6c2ff63 P b025089f6b444f24bab9c179513111b0: Deleting consensus metadata
I20260809 12:08:29.696117 26863 ts_tablet_manager.cc:1918] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:29.696344 26863 log.cc:1196] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/d49c67ce2bd047dd9cb3497eb6c2ff63
I20260809 12:08:29.696972 26863 ts_tablet_manager.cc:1939] T d49c67ce2bd047dd9cb3497eb6c2ff63 P 677871a664f542ad9a2bddfa23352687: Deleting consensus metadata
I20260809 12:08:29.698110 26135 catalog_manager.cc:4841] TS b025089f6b444f24bab9c179513111b0 (127.0.147.67:43953): tablet d49c67ce2bd047dd9cb3497eb6c2ff63 (table test-table [id=1dc5bbe27c1f4fa9a7d4ea54bc82e7ba]) successfully deleted
I20260809 12:08:29.698794 26135 catalog_manager.cc:4841] TS 677871a664f542ad9a2bddfa23352687 (127.0.147.66:41771): tablet d49c67ce2bd047dd9cb3497eb6c2ff63 (table test-table [id=1dc5bbe27c1f4fa9a7d4ea54bc82e7ba]) successfully deleted
I20260809 12:08:29.789006 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 26239
W20260809 12:08:29.821219 26135 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.147.65:39943 (error 108)
I20260809 12:08:29.821492 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 26428
I20260809 12:08:29.823684 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.823104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.0.1:37244)
0809 12:08:29.823315 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.823475 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 12:08:29.824005 26135 catalog_manager.cc:4584] TS a84fba94eb7a40aa88cdc33a34ac880e (127.0.147.65:39943): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d49c67ce2bd047dd9cb3497eb6c2ff63: Network error: Client connection negotiation failed: client connection to 127.0.147.65:39943: connect: Connection refused (error 111)
I20260809 12:08:29.830503 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.823135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.0.147.66:41771)
0809 12:08:29.823338 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.823344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.823367 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:08:29.823441 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.823448 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.823593 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.823715 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:29.824346 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.825171 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.827103 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.827806 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.827914 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.829095 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:29.829103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:29.829117 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:29.829750 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:29.829797 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:29.829817 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:29.829908 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.830123 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.830208 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:08:29.831241 26609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:29.877770 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.877218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.0.1:37246)
0809 12:08:29.877451 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.877577 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 12:08:29.878983 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.878529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.0.1:37262)
0809 12:08:29.878707 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.878794 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 12:08:29.939896 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.939284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.0.1:37266)
0809 12:08:29.939609 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.939713 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 12:08:29.940965 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.940645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.0.1:37272)
0809 12:08:29.940777 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:29.940846 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 12:08:29.955111 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 26428
I20260809 12:08:29.983827 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 26618
I20260809 12:08:29.991101 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:29.984690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.0.147.67:43953)
0809 12:08:29.984910 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:29.984919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:29.985046 (+ 127us) server_negotiation.cc:407] Connection header received
0809 12:08:29.985250 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:29.985256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:29.985351 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:29.985467 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:29.985909 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.986657 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.988072 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:29.988461 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:29.988601 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:29.989758 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:29.989767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:29.989781 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:29.990341 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:29.990390 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:29.990409 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:29.990536 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:29.990693 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 12:08:29.990803 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:08:29.991971 26789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:30.021785 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:30.021306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.0.1:37284)
0809 12:08:30.021524 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:30.021629 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 12:08:30.022861 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:30.022468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.0.1:37300)
0809 12:08:30.022676 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:30.022748 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 12:08:30.083104 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 26618
I20260809 12:08:30.109654 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 26115
I20260809 12:08:30.110824 26200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:30.223373 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 26115
I20260809 12:08:30.223443 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:30.222624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:39943 (local address 127.0.0.1:37302)
0809 12:08:30.223151 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:30.223256 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358}
2026-08-09T12:08:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2905 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:08:30 dist-test-slave-sb2z krb5kdc[26875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:08:30 dist-test-slave-sb2z krb5kdc[26875](info): set up 2 sockets
Aug 09 12:08:30 dist-test-slave-sb2z krb5kdc[26875](info): commencing operation
krb5kdc: starting...
W20260809 12:08:31.575840 589 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.006s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:08:31 dist-test-slave-sb2z krb5kdc[26875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:08:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:08:31.900964 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45011
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:45011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:32.247512 26894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:32.247821 26894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:32.247915 26894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:32.254870 26894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:32.254972 26894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:32.255039 26894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:32.255122 26894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:32.255172 26894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:32.265309 26894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:45011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:32.266956 26894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:32.268186 26894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:32.280012 26900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:32.280081 26901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:32.280135 26904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:32.284905 26894 server_base.cc:1034] running on GCE node
I20260809 12:08:32.285686 26894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:32.286972 26894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:32.288182 26894 hybrid_clock.cc:648] HybridClock initialized: now 1786277312288146 us; error 34 us; skew 500 ppm
I20260809 12:08:32.288534 26894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:32 dist-test-slave-sb2z krb5kdc[26875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277312, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:32.291694 26894 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:08:32.293278 26894 webserver.cc:460] Webserver started at http://127.0.147.126:37071/ using document root <none> and password file <none>
I20260809 12:08:32.293977 26894 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:32.294137 26894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:32.294469 26894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:32.297498 26894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "9fc990f55009499fa6390cb078ce8559"
format_stamp: "Formatted at 2026-08-09 12:08:32 on dist-test-slave-sb2z"
server_key: "2d67917cfa5b01294c750f3ef2afa13c"
server_key_iv: "66fa1b0c1d6500cf52c7b2ef844ba09b"
server_key_version: "encryptionkey@0"
I20260809 12:08:32.298269 26894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "9fc990f55009499fa6390cb078ce8559"
format_stamp: "Formatted at 2026-08-09 12:08:32 on dist-test-slave-sb2z"
server_key: "2d67917cfa5b01294c750f3ef2afa13c"
server_key_iv: "66fa1b0c1d6500cf52c7b2ef844ba09b"
server_key_version: "encryptionkey@0"
I20260809 12:08:32.303486 26894 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:08:32.306878 26912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:32.308166 26894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:08:32.308351 26894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "9fc990f55009499fa6390cb078ce8559"
format_stamp: "Formatted at 2026-08-09 12:08:32 on dist-test-slave-sb2z"
server_key: "2d67917cfa5b01294c750f3ef2afa13c"
server_key_iv: "66fa1b0c1d6500cf52c7b2ef844ba09b"
server_key_version: "encryptionkey@0"
I20260809 12:08:32.308547 26894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:32.326586 26894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:32.330457 26894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:32.330741 26894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:32.349962 26894 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:45011
I20260809 12:08:32.349993 26991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:45011 every 8 connection(s)
I20260809 12:08:32.351305 26894 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:32.352696 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 26894
I20260809 12:08:32.352869 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:32.353420 589 external_mini_cluster.cc:1428] Setting key 074dbb56d0712b03665f2514d8858b16
I20260809 12:08:32.355952 26993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:08:32 dist-test-slave-sb2z krb5kdc[26875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:32.369400 26993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559: Bootstrap starting.
I20260809 12:08:32.372709 26993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:32.373756 26993 log.cc:824] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:32.377063 26993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559: No bootstrap required, opened a new log
I20260809 12:08:32.377677 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:32.355318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.0.147.126:45011)
0809 12:08:32.356162 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:32.356173 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:32.356212 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:08:32.357897 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:32.357920 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:32.358281 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:32.358607 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:32.360491 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:32.361831 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:32.363761 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:32.364396 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:32.364532 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:32.369668 (+ 5136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:32.369675 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:32.369689 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:32.369736 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:32.372996 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:32.373769 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:32.373775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:32.373782 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:32.374036 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:32.374518 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:32.374521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:32.374523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:32.375147 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:32.375434 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:32.375870 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 12:08:32.376322 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":634,"thread_start_us":428,"threads_started":1}
I20260809 12:08:32.384704 26993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc990f55009499fa6390cb078ce8559" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45011 } }
I20260809 12:08:32.385018 26993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:32.385095 26993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc990f55009499fa6390cb078ce8559, State: Initialized, Role: FOLLOWER
I20260809 12:08:32.385629 26993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc990f55009499fa6390cb078ce8559" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45011 } }
I20260809 12:08:32.385836 26993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:32.385931 26993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:32.386057 26993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:32.388168 26993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc990f55009499fa6390cb078ce8559" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45011 } }
I20260809 12:08:32.388597 26993 leader_election.cc:302] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fc990f55009499fa6390cb078ce8559; no voters:
I20260809 12:08:32.389158 26993 leader_election.cc:288] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:32.389313 27001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:32.389981 27001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [term 1 LEADER]: Becoming Leader. State: Replica: 9fc990f55009499fa6390cb078ce8559, State: Running, Role: LEADER
I20260809 12:08:32.390614 27001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc990f55009499fa6390cb078ce8559" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45011 } }
I20260809 12:08:32.391360 26993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:32.393637 27004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fc990f55009499fa6390cb078ce8559. Latest consensus state: current_term: 1 leader_uuid: "9fc990f55009499fa6390cb078ce8559" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc990f55009499fa6390cb078ce8559" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45011 } } }
I20260809 12:08:32.393669 27002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fc990f55009499fa6390cb078ce8559" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc990f55009499fa6390cb078ce8559" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45011 } } }
I20260809 12:08:32.394098 27004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:32.394098 27002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:32.397205 27016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:32.401360 27016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:32.409731 27016 catalog_manager.cc:1347] Generated new cluster ID: b3b75d0b1e36422bb75ad1d677a8db33
I20260809 12:08:32.409855 27016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:32.429491 27016 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:32.430792 27016 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:32.436635 27016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559: Generated new TSK 0
I20260809 12:08:32.437379 27016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:08:32.519021 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45011
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:32.864269 27033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:32.864527 27033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:32.864711 27033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:32.871805 27033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:32.871933 27033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:32.872124 27033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:08:32.881863 27033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:32.883085 27033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:32.884497 27033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:32.897941 27040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:32.898006 27041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:32.897884 27043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:32.899039 27033 server_base.cc:1034] running on GCE node
I20260809 12:08:32.899653 27033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:32.900502 27033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:32.901710 27033 hybrid_clock.cc:648] HybridClock initialized: now 1786277312901674 us; error 26 us; skew 500 ppm
I20260809 12:08:32.902022 27033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:32 dist-test-slave-sb2z krb5kdc[26875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277312, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:32.910161 27033 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:08:32.911882 27033 webserver.cc:460] Webserver started at http://127.0.147.65:44445/ using document root <none> and password file <none>
I20260809 12:08:32.912833 27033 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:32.913003 27033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:32.913385 27033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:32.916422 27033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "11083a829bee46ba9270c855318a13bb"
format_stamp: "Formatted at 2026-08-09 12:08:32 on dist-test-slave-sb2z"
server_key: "e4c28fb175968b5cacb2444de2dba5f8"
server_key_iv: "3ba927f9375ead6272f7eef5cc6f4907"
server_key_version: "encryptionkey@0"
I20260809 12:08:32.917238 27033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "11083a829bee46ba9270c855318a13bb"
format_stamp: "Formatted at 2026-08-09 12:08:32 on dist-test-slave-sb2z"
server_key: "e4c28fb175968b5cacb2444de2dba5f8"
server_key_iv: "3ba927f9375ead6272f7eef5cc6f4907"
server_key_version: "encryptionkey@0"
I20260809 12:08:32.922453 27033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:08:32.925930 27052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:32.927255 27033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:08:32.927430 27033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "11083a829bee46ba9270c855318a13bb"
format_stamp: "Formatted at 2026-08-09 12:08:32 on dist-test-slave-sb2z"
server_key: "e4c28fb175968b5cacb2444de2dba5f8"
server_key_iv: "3ba927f9375ead6272f7eef5cc6f4907"
server_key_version: "encryptionkey@0"
I20260809 12:08:32.927702 27033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:32.954319 27033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:32.958441 27033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:32.958775 27033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:32.959800 27033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:32.961153 27033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:32.961259 27033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:32.961409 27033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:32.961483 27033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:32.995995 27033 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:41747
I20260809 12:08:32.996057 27212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:41747 every 8 connection(s)
I20260809 12:08:32.997303 27033 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:08:33.003283 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 27033
I20260809 12:08:33.003482 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:08:33.003966 589 external_mini_cluster.cc:1428] Setting key cee8a59b5fbca17686986e67c8f18fd2
Aug 09 12:08:33 dist-test-slave-sb2z krb5kdc[26875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277312, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:33.020418 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:32.999795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:52217 (local address 127.0.147.126:45011)
0809 12:08:33.000290 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:33.000294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:33.001453 (+ 1159us) server_negotiation.cc:407] Connection header received
0809 12:08:33.003345 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:33.003350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:33.003441 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:33.003624 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:33.005365 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:33.006162 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:33.007782 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:33.008187 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:33.008335 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:33.012895 (+ 4560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:33.012904 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:33.012911 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:33.012954 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:33.016724 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:33.017797 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:33.017806 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:33.017812 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:33.017977 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:33.018404 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:33.018412 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:33.018418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:33.018808 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:33.018965 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:33.019661 (+ 696us) server_negotiation.cc:299] Negotiation successful
0809 12:08:33.019918 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":285,"threads_started":1}
I20260809 12:08:33.021214 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:33.000472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45011 (local address 127.0.147.65:52217)
0809 12:08:33.001279 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:33.001323 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:33.003064 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:33.003865 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:33.003873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:33.004465 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:33.005184 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:33.005207 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:33.006341 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:33.006346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:33.007636 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:33.007650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:33.008428 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:33.008437 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:33.008679 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:33.009491 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:08:33.009528 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:33.012647 (+ 3119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:33.016943 (+ 4296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:33.016952 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:33.016970 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:33.017645 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:33.018091 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:33.018095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:33.018097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:33.018283 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:33.018938 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:33.018943 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:33.019253 (+ 310us) client_negotiation.cc:769] Sending connection context
0809 12:08:33.019481 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 12:08:33.019953 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":298,"threads_started":1}
I20260809 12:08:33.022922 27213 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45011
I20260809 12:08:33.023279 27213 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:33.024051 27213 heartbeater.cc:511] Master 127.0.147.126:45011 requested a full tablet report, sending...
I20260809 12:08:33.026126 26939 ts_manager.cc:194] Registered new tserver with Master: 11083a829bee46ba9270c855318a13bb (127.0.147.65:41747)
I20260809 12:08:33.027642 26939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:52217
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:08:33.076963 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45011
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:33.423302 27223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:33.423643 27223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:33.423760 27223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:33.430794 27223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:33.430898 27223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:33.431142 27223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:08:33.441057 27223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:33.442358 27223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:33.443801 27223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:33.456972 27232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:33.457044 27234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:33.457020 27231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:33.458048 27223 server_base.cc:1034] running on GCE node
I20260809 12:08:33.458710 27223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:33.459699 27223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:33.460942 27223 hybrid_clock.cc:648] HybridClock initialized: now 1786277313460869 us; error 30 us; skew 500 ppm
I20260809 12:08:33.461264 27223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:33 dist-test-slave-sb2z krb5kdc[26875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277313, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:33.464633 27223 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:08:33.466239 27223 webserver.cc:460] Webserver started at http://127.0.147.66:42193/ using document root <none> and password file <none>
I20260809 12:08:33.466969 27223 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:33.467111 27223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:33.467449 27223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:33.470458 27223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "85ff5e3a3a84437f952d1138a8bebcf8"
format_stamp: "Formatted at 2026-08-09 12:08:33 on dist-test-slave-sb2z"
server_key: "1ff3141188179adc44dd64f7ab25811c"
server_key_iv: "40cd8512ec20d372770a4e86f54db5c7"
server_key_version: "encryptionkey@0"
I20260809 12:08:33.471271 27223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "85ff5e3a3a84437f952d1138a8bebcf8"
format_stamp: "Formatted at 2026-08-09 12:08:33 on dist-test-slave-sb2z"
server_key: "1ff3141188179adc44dd64f7ab25811c"
server_key_iv: "40cd8512ec20d372770a4e86f54db5c7"
server_key_version: "encryptionkey@0"
I20260809 12:08:33.476428 27223 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 12:08:33.479867 27243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:33.481199 27223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:08:33.481474 27223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "85ff5e3a3a84437f952d1138a8bebcf8"
format_stamp: "Formatted at 2026-08-09 12:08:33 on dist-test-slave-sb2z"
server_key: "1ff3141188179adc44dd64f7ab25811c"
server_key_iv: "40cd8512ec20d372770a4e86f54db5c7"
server_key_version: "encryptionkey@0"
I20260809 12:08:33.481693 27223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:33.496455 27223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:33.500716 27223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:33.501035 27223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:33.502008 27223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:33.503348 27223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:33.503458 27223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:33.503636 27223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:33.503719 27223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:33.538906 27223 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:33443
I20260809 12:08:33.538962 27405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:33443 every 8 connection(s)
I20260809 12:08:33.540343 27223 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:08:33.550647 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 27223
I20260809 12:08:33.550851 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:08:33.551453 589 external_mini_cluster.cc:1428] Setting key 35d93e3ba23db0f66ef74edd810fab36
Aug 09 12:08:33 dist-test-slave-sb2z krb5kdc[26875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277313, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:33.560631 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:33.542913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:44721 (local address 127.0.147.126:45011)
0809 12:08:33.543382 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:33.543388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:33.544484 (+ 1096us) server_negotiation.cc:407] Connection header received
0809 12:08:33.546392 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:33.546396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:33.546467 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:33.546572 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:33.548124 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:33.548851 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:33.550480 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:33.551000 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:33.551109 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:33.554781 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:33.554787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:33.554790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:33.554821 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:33.557450 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:33.558195 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:33.558199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:33.558202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:33.558331 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:33.558786 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:33.558791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:33.558793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:33.559052 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:33.559216 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:33.559968 (+ 752us) server_negotiation.cc:299] Negotiation successful
0809 12:08:33.560242 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":234,"threads_started":1}
I20260809 12:08:33.561546 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:33.543597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45011 (local address 127.0.147.66:44721)
0809 12:08:33.544324 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:33.544362 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:33.546137 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:33.546779 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:33.546787 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:33.547322 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:33.547945 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:33.547961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:33.548985 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:33.549022 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:33.550319 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:33.550338 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:33.551229 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:33.551236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:33.551572 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:33.552357 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:08:33.552380 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:33.554543 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:33.557607 (+ 3064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:33.557616 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:33.557631 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:33.558048 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:33.558449 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:33.558454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:33.558458 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:33.558657 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:33.559184 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:33.559192 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:33.559565 (+ 373us) client_negotiation.cc:769] Sending connection context
0809 12:08:33.559849 (+ 284us) client_negotiation.cc:240] Negotiation successful
0809 12:08:33.560432 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":270,"threads_started":1}
I20260809 12:08:33.563095 27407 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45011
I20260809 12:08:33.563402 27407 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:33.564144 27407 heartbeater.cc:511] Master 127.0.147.126:45011 requested a full tablet report, sending...
I20260809 12:08:33.565665 26939 ts_manager.cc:194] Registered new tserver with Master: 85ff5e3a3a84437f952d1138a8bebcf8 (127.0.147.66:33443)
I20260809 12:08:33.566915 26939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:44721
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:08:33.624836 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45011
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:33.962805 27415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:33.963115 27415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:33.963297 27415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:33.970485 27415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:33.970664 27415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:33.970856 27415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:08:33.980527 27415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:33.981803 27415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:33.983101 27415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:33.996596 27423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:33.996836 27422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:33.997690 27425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:33.998137 27415 server_base.cc:1034] running on GCE node
I20260809 12:08:33.998772 27415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:33.999841 27415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:34.001071 27415 hybrid_clock.cc:648] HybridClock initialized: now 1786277314001025 us; error 39 us; skew 500 ppm
I20260809 12:08:34.001392 27415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:34 dist-test-slave-sb2z krb5kdc[26875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277314, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:34.004492 27415 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:08:34.005822 27415 webserver.cc:460] Webserver started at http://127.0.147.67:41547/ using document root <none> and password file <none>
I20260809 12:08:34.006465 27415 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:34.006570 27415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:34.006824 27415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:34.009686 27415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "c3f7690259f54025bf43d04ab10c4da5"
format_stamp: "Formatted at 2026-08-09 12:08:34 on dist-test-slave-sb2z"
server_key: "9e6404c1c3dd7e2d78698e9c708a9079"
server_key_iv: "5a7a59c84b59e62fdd6974d4a40d2335"
server_key_version: "encryptionkey@0"
I20260809 12:08:34.010433 27415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "c3f7690259f54025bf43d04ab10c4da5"
format_stamp: "Formatted at 2026-08-09 12:08:34 on dist-test-slave-sb2z"
server_key: "9e6404c1c3dd7e2d78698e9c708a9079"
server_key_iv: "5a7a59c84b59e62fdd6974d4a40d2335"
server_key_version: "encryptionkey@0"
I20260809 12:08:34.015451 27415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:08:34.019271 27435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:34.021581 27415 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260809 12:08:34.021728 27415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "c3f7690259f54025bf43d04ab10c4da5"
format_stamp: "Formatted at 2026-08-09 12:08:34 on dist-test-slave-sb2z"
server_key: "9e6404c1c3dd7e2d78698e9c708a9079"
server_key_iv: "5a7a59c84b59e62fdd6974d4a40d2335"
server_key_version: "encryptionkey@0"
I20260809 12:08:34.021907 27415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:34.030462 27213 heartbeater.cc:503] Master 127.0.147.126:45011 was elected leader, sending a full tablet report...
I20260809 12:08:34.057824 27415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:34.062310 27415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:34.062587 27415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:34.063597 27415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:34.064919 27415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:34.064997 27415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:34.065095 27415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:34.065142 27415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:34.099506 27415 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:33895
I20260809 12:08:34.099617 27596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:33895 every 8 connection(s)
I20260809 12:08:34.100881 27415 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:08:34.106365 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 27415
I20260809 12:08:34.106554 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:08:34.107012 589 external_mini_cluster.cc:1428] Setting key b44e2eebe9f754075243a4b65aa0ba53
Aug 09 12:08:34 dist-test-slave-sb2z krb5kdc[26875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277314, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:34.121712 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.103616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:57985 (local address 127.0.147.126:45011)
0809 12:08:34.104153 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.104158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.104916 (+ 758us) server_negotiation.cc:407] Connection header received
0809 12:08:34.106954 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.106959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.107073 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.107210 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:34.109016 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.110267 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.112010 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.112513 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.112635 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.116149 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:34.116153 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:34.116157 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:34.116188 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:34.118709 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.119374 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.119378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.119380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.119500 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.119953 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.119956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.119958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.120214 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:34.120334 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.121059 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.121289 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":263,"threads_started":1}
I20260809 12:08:34.122709 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.104058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45011 (local address 127.0.147.67:57985)
0809 12:08:34.104749 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:34.104789 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:34.106662 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:34.107501 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:34.107513 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:34.108088 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:34.108776 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.108807 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.110442 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.110446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.111851 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.111863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.112730 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.112736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.112936 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.113688 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:08:34.113716 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:34.115943 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:34.118829 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:34.118836 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:34.118847 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:34.119266 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:34.119665 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:34.119669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:34.119671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:34.119851 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:34.120321 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:34.120326 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:34.120642 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 12:08:34.120951 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 12:08:34.121482 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":280,"threads_started":1}
I20260809 12:08:34.124185 27597 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45011
I20260809 12:08:34.124491 27597 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:34.125211 27597 heartbeater.cc:511] Master 127.0.147.126:45011 requested a full tablet report, sending...
I20260809 12:08:34.126741 26939 ts_manager.cc:194] Registered new tserver with Master: c3f7690259f54025bf43d04ab10c4da5 (127.0.147.67:33895)
I20260809 12:08:34.127617 26939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:57985
I20260809 12:08:34.134970 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:08:34.151788 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.139117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39232 (local address 127.0.147.126:45011)
0809 12:08:34.139274 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.139278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.139777 (+ 499us) server_negotiation.cc:407] Connection header received
0809 12:08:34.139908 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.139911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.139972 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.140062 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:34.141572 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.142293 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.143518 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.143911 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.144001 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.146530 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:34.146535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:34.146538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:34.146569 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:34.149366 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.150045 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.150048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.150051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.150175 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.150555 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.150558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.150560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.150847 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:34.151060 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.151365 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.151498 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 12:08:34.155692 26939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:08:34.162588 26939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:08:34.192564 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.176189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:41747 (local address 127.0.0.1:46848)
0809 12:08:34.184487 (+ 8298us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:34.184510 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:34.184623 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:34.185336 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:34.185349 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:34.185377 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:34.185853 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.185862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.188178 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.188187 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.189844 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.189860 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.191466 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.191473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.191685 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.191800 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 12:08:34.191941 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 12:08:34.192141 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8109,"thread_start_us":7978,"threads_started":1}
I20260809 12:08:34.192866 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.175171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.0.147.67:33895)
0809 12:08:34.183462 (+ 8291us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.183475 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.184173 (+ 698us) server_negotiation.cc:407] Connection header received
0809 12:08:34.184298 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.184305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.184587 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.184832 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:34.185568 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.187200 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.189973 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.191599 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.191716 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.191979 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.192208 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.192368 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8071,"thread_start_us":7881,"threads_started":1}
I20260809 12:08:34.193092 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.176809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.0.147.65:41747)
0809 12:08:34.184683 (+ 7874us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.184693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.184721 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:08:34.184826 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.184833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.185068 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.185272 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:34.186040 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.188050 (+ 2010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.190037 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.191258 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.191415 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.191691 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.192625 (+ 934us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.192790 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7697,"thread_start_us":6927,"threads_started":1}
I20260809 12:08:34.193284 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.174661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:33895 (local address 127.0.0.1:60132)
0809 12:08:34.183871 (+ 9210us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:34.183890 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:34.184016 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:34.184869 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:34.184876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:34.184907 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:34.185296 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.185310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.187346 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.187352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.189753 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.189771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.191851 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.191857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.192022 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.192052 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 12:08:34.192137 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 12:08:34.192240 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9078,"thread_start_us":6565,"threads_started":1}
I20260809 12:08:34.194036 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.174579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:33443 (local address 127.0.0.1:60128)
0809 12:08:34.181749 (+ 7170us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:34.181805 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:34.182032 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:34.185456 (+ 3424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:34.185460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:34.185492 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:34.186032 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.186048 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.188124 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.188129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.191296 (+ 3167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.191311 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.193244 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.193251 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.193448 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.193483 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:08:34.193552 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 12:08:34.193690 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6812,"mutex_wait_us":1,"thread_start_us":8949,"threads_started":1}
I20260809 12:08:34.194588 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.174620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60128 (local address 127.0.147.66:33443)
0809 12:08:34.184826 (+ 10206us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.184835 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.184862 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:08:34.184958 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.184967 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.185225 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.186116 (+ 891us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:34.186372 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.187963 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.191495 (+ 3532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.192981 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.193810 (+ 829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.193989 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.194053 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.194207 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9997,"thread_start_us":10439,"threads_started":1}
I20260809 12:08:34.197429 27503 tablet_service.cc:1467] Processing CreateTablet for tablet 20451ef3713a4866922852d05d1a7fd1 (DEFAULT_TABLE table=test-table [id=8e524910008c4c6796df484f5b8c9b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:34.197429 27120 tablet_service.cc:1467] Processing CreateTablet for tablet 20451ef3713a4866922852d05d1a7fd1 (DEFAULT_TABLE table=test-table [id=8e524910008c4c6796df484f5b8c9b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:34.197773 27312 tablet_service.cc:1467] Processing CreateTablet for tablet 20451ef3713a4866922852d05d1a7fd1 (DEFAULT_TABLE table=test-table [id=8e524910008c4c6796df484f5b8c9b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:34.198916 27312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20451ef3713a4866922852d05d1a7fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:34.198926 27120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20451ef3713a4866922852d05d1a7fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:34.198941 27503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20451ef3713a4866922852d05d1a7fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:34.210542 27622 tablet_bootstrap.cc:492] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: Bootstrap starting.
I20260809 12:08:34.214404 27622 tablet_bootstrap.cc:654] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:34.215442 27622 log.cc:824] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:34.218369 27622 tablet_bootstrap.cc:492] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: No bootstrap required, opened a new log
I20260809 12:08:34.218662 27622 ts_tablet_manager.cc:1397] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 12:08:34.220299 27625 tablet_bootstrap.cc:492] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: Bootstrap starting.
I20260809 12:08:34.220453 27626 tablet_bootstrap.cc:492] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: Bootstrap starting.
I20260809 12:08:34.223680 27626 tablet_bootstrap.cc:654] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:34.224293 27625 tablet_bootstrap.cc:654] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:34.224835 27626 log.cc:824] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:34.225574 27625 log.cc:824] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:34.227818 27626 tablet_bootstrap.cc:492] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: No bootstrap required, opened a new log
I20260809 12:08:34.228121 27626 ts_tablet_manager.cc:1397] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.005s
I20260809 12:08:34.229586 27622 raft_consensus.cc:348] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } }
I20260809 12:08:34.230051 27622 raft_consensus.cc:374] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:34.230116 27625 tablet_bootstrap.cc:492] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: No bootstrap required, opened a new log
I20260809 12:08:34.230170 27622 raft_consensus.cc:729] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85ff5e3a3a84437f952d1138a8bebcf8, State: Initialized, Role: FOLLOWER
I20260809 12:08:34.230371 27625 ts_tablet_manager.cc:1397] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 12:08:34.230825 27622 consensus_queue.cc:260] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } }
I20260809 12:08:34.232066 27407 heartbeater.cc:503] Master 127.0.147.126:45011 was elected leader, sending a full tablet report...
I20260809 12:08:34.234112 27622 ts_tablet_manager.cc:1428] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
I20260809 12:08:34.237475 27626 raft_consensus.cc:348] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } }
I20260809 12:08:34.237914 27626 raft_consensus.cc:374] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:34.238035 27626 raft_consensus.cc:729] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11083a829bee46ba9270c855318a13bb, State: Initialized, Role: FOLLOWER
I20260809 12:08:34.238651 27626 consensus_queue.cc:260] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } }
I20260809 12:08:34.239866 27625 raft_consensus.cc:348] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } }
I20260809 12:08:34.240235 27625 raft_consensus.cc:374] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:34.240249 27626 ts_tablet_manager.cc:1428] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260809 12:08:34.240355 27625 raft_consensus.cc:729] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f7690259f54025bf43d04ab10c4da5, State: Initialized, Role: FOLLOWER
I20260809 12:08:34.240871 27625 consensus_queue.cc:260] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } }
I20260809 12:08:34.242205 27597 heartbeater.cc:503] Master 127.0.147.126:45011 was elected leader, sending a full tablet report...
I20260809 12:08:34.242517 27625 ts_tablet_manager.cc:1428] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260809 12:08:34.249688 27214 tablet.cc:2369] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:08:34.292415 27408 tablet.cc:2369] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:08:34.352815 27598 tablet.cc:2369] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:08:34.454473 27635 raft_consensus.cc:482] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:08:34.454752 27635 raft_consensus.cc:504] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } }
I20260809 12:08:34.456583 27635 leader_election.cc:288] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3f7690259f54025bf43d04ab10c4da5 (127.0.147.67:33895), 11083a829bee46ba9270c855318a13bb (127.0.147.65:41747)
I20260809 12:08:34.463704 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.457032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:43651 (local address 127.0.147.67:33895)
0809 12:08:34.457336 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.457345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.458249 (+ 904us) server_negotiation.cc:407] Connection header received
0809 12:08:34.458459 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.458463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.458529 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.458628 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:34.459271 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.460103 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.461900 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.462662 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.462874 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.462957 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.463275 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.463404 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:08:34.463711 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.457375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:36197 (local address 127.0.147.65:41747)
0809 12:08:34.457654 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.457663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.457829 (+ 166us) server_negotiation.cc:407] Connection header received
0809 12:08:34.458100 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.458301 (+ 201us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.458390 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.458538 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:34.459199 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.460016 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.461891 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.462701 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.462887 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.462983 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.463302 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.463428 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:08:34.463889 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.456895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:33895 (local address 127.0.147.66:43651)
0809 12:08:34.458142 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:34.458156 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:34.458301 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:34.458650 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:34.458654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:34.458694 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:34.459113 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.459123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.460262 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.460269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.461735 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.461747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.462916 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.462926 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.463110 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.463135 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:08:34.463246 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 12:08:34.463379 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1090,"mutex_wait_us":12,"thread_start_us":883,"threads_started":1}
I20260809 12:08:34.463918 27640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.456887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:41747 (local address 127.0.147.66:36197)
0809 12:08:34.457708 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:34.457733 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:34.457923 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:34.458555 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:34.458560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:34.458585 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:34.459041 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.459052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.460157 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.460163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.461736 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:34.461748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.462943 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.462949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:34.463135 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.463158 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:08:34.463262 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 12:08:34.463385 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":30,"thread_start_us":543,"threads_started":1}
I20260809 12:08:34.465063 27539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20451ef3713a4866922852d05d1a7fd1" candidate_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7690259f54025bf43d04ab10c4da5" is_pre_election: true
I20260809 12:08:34.465066 27148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20451ef3713a4866922852d05d1a7fd1" candidate_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11083a829bee46ba9270c855318a13bb" is_pre_election: true
I20260809 12:08:34.465596 27539 raft_consensus.cc:2454] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85ff5e3a3a84437f952d1138a8bebcf8 in term 0.
I20260809 12:08:34.465592 27148 raft_consensus.cc:2454] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85ff5e3a3a84437f952d1138a8bebcf8 in term 0.
I20260809 12:08:34.466270 27250 leader_election.cc:302] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11083a829bee46ba9270c855318a13bb, 85ff5e3a3a84437f952d1138a8bebcf8; no voters:
I20260809 12:08:34.466702 27635 raft_consensus.cc:2781] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:08:34.466830 27635 raft_consensus.cc:482] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:08:34.466903 27635 raft_consensus.cc:3037] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:34.469203 27635 raft_consensus.cc:504] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } }
I20260809 12:08:34.469818 27635 leader_election.cc:288] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [CANDIDATE]: Term 1 election: Requested vote from peers c3f7690259f54025bf43d04ab10c4da5 (127.0.147.67:33895), 11083a829bee46ba9270c855318a13bb (127.0.147.65:41747)
I20260809 12:08:34.470292 27539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20451ef3713a4866922852d05d1a7fd1" candidate_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7690259f54025bf43d04ab10c4da5"
I20260809 12:08:34.470343 27148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20451ef3713a4866922852d05d1a7fd1" candidate_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11083a829bee46ba9270c855318a13bb"
I20260809 12:08:34.470539 27539 raft_consensus.cc:3037] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:34.470539 27148 raft_consensus.cc:3037] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:34.473858 27539 raft_consensus.cc:2454] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85ff5e3a3a84437f952d1138a8bebcf8 in term 1.
I20260809 12:08:34.473858 27148 raft_consensus.cc:2454] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85ff5e3a3a84437f952d1138a8bebcf8 in term 1.
I20260809 12:08:34.474347 27244 leader_election.cc:302] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 11083a829bee46ba9270c855318a13bb, 85ff5e3a3a84437f952d1138a8bebcf8, c3f7690259f54025bf43d04ab10c4da5; no voters:
I20260809 12:08:34.474714 27635 raft_consensus.cc:2781] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:34.475288 27635 raft_consensus.cc:686] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 1 LEADER]: Becoming Leader. State: Replica: 85ff5e3a3a84437f952d1138a8bebcf8, State: Running, Role: LEADER
I20260809 12:08:34.475905 27635 consensus_queue.cc:237] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } }
I20260809 12:08:34.480423 26939 catalog_manager.cc:5495] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85ff5e3a3a84437f952d1138a8bebcf8 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ff5e3a3a84437f952d1138a8bebcf8" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 } health_report { overall_health: UNKNOWN } } }
I20260809 12:08:34.560357 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.554843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.0.147.66:33443)
0809 12:08:34.555069 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.555077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.555137 (+ 60us) server_negotiation.cc:407] Connection header received
0809 12:08:34.555333 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.555337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.555416 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.555627 (+ 211us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:08:34.555959 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.556775 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.558131 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.558553 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.558663 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.559159 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:08:34.559737 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:08:34.559872 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.559988 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.560026 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:08:34.568714 27539 raft_consensus.cc:1261] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [term 1 FOLLOWER]: Refusing update from remote peer 85ff5e3a3a84437f952d1138a8bebcf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:34.568714 27148 raft_consensus.cc:1261] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [term 1 FOLLOWER]: Refusing update from remote peer 85ff5e3a3a84437f952d1138a8bebcf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:34.569904 27642 consensus_queue.cc:1035] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11083a829bee46ba9270c855318a13bb" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 41747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:34.570268 27635 consensus_queue.cc:1035] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3f7690259f54025bf43d04ab10c4da5" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:34.594385 26937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:34.594681 26937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:34.598531 26937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fc990f55009499fa6390cb078ce8559: Sending DeleteTablet for 3 replicas of tablet 20451ef3713a4866922852d05d1a7fd1
I20260809 12:08:34.599756 27120 tablet_service.cc:1514] Processing DeleteTablet for tablet 20451ef3713a4866922852d05d1a7fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:34 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:46848
I20260809 12:08:34.599768 27312 tablet_service.cc:1514] Processing DeleteTablet for tablet 20451ef3713a4866922852d05d1a7fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:34 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:60128
I20260809 12:08:34.599783 27503 tablet_service.cc:1514] Processing DeleteTablet for tablet 20451ef3713a4866922852d05d1a7fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:34 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:60132
I20260809 12:08:34.602200 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 27033
I20260809 12:08:34.611496 27658 tablet_replica.cc:330] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: stopping tablet replica
I20260809 12:08:34.612239 27658 raft_consensus.cc:2229] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:34.613003 27658 raft_consensus.cc:2258] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:34.614179 27659 tablet_replica.cc:330] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: stopping tablet replica
I20260809 12:08:34.614398 27660 tablet_replica.cc:330] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: stopping tablet replica
I20260809 12:08:34.614591 27659 raft_consensus.cc:2229] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:08:34.614838 27660 raft_consensus.cc:2229] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:34.615329 27660 raft_consensus.cc:2258] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:34.616255 27658 ts_tablet_manager.cc:1905] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:34.617398 27660 ts_tablet_manager.cc:1905] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:34.623382 27658 ts_tablet_manager.cc:1918] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:34.624120 27659 raft_consensus.cc:2258] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:34.624625 27660 ts_tablet_manager.cc:1918] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:34.624832 27660 log.cc:1196] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/20451ef3713a4866922852d05d1a7fd1
I20260809 12:08:34.625391 27660 ts_tablet_manager.cc:1939] T 20451ef3713a4866922852d05d1a7fd1 P c3f7690259f54025bf43d04ab10c4da5: Deleting consensus metadata
I20260809 12:08:34.626483 27659 ts_tablet_manager.cc:1905] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:34.627947 26913 catalog_manager.cc:4841] TS c3f7690259f54025bf43d04ab10c4da5 (127.0.147.67:33895): tablet 20451ef3713a4866922852d05d1a7fd1 (table test-table [id=8e524910008c4c6796df484f5b8c9b1a]) successfully deleted
Aug 09 12:08:34 dist-test-slave-sb2z krb5kdc[26875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:08:34.633051 27659 ts_tablet_manager.cc:1918] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:34.633232 27659 log.cc:1196] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/20451ef3713a4866922852d05d1a7fd1
I20260809 12:08:34.633805 27659 ts_tablet_manager.cc:1939] T 20451ef3713a4866922852d05d1a7fd1 P 85ff5e3a3a84437f952d1138a8bebcf8: Deleting consensus metadata
I20260809 12:08:34.635252 27658 log.cc:1196] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/20451ef3713a4866922852d05d1a7fd1
I20260809 12:08:34.635295 26915 catalog_manager.cc:4841] TS 85ff5e3a3a84437f952d1138a8bebcf8 (127.0.147.66:33443): tablet 20451ef3713a4866922852d05d1a7fd1 (table test-table [id=8e524910008c4c6796df484f5b8c9b1a]) successfully deleted
I20260809 12:08:34.635926 27658 ts_tablet_manager.cc:1939] T 20451ef3713a4866922852d05d1a7fd1 P 11083a829bee46ba9270c855318a13bb: Deleting consensus metadata
I20260809 12:08:34.637635 26917 catalog_manager.cc:4841] TS 11083a829bee46ba9270c855318a13bb (127.0.147.65:41747): tablet 20451ef3713a4866922852d05d1a7fd1 (table test-table [id=8e524910008c4c6796df484f5b8c9b1a]) successfully deleted
I20260809 12:08:34.639070 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.603635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.0.147.65:41747)
0809 12:08:34.603818 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.603824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.618975 (+ 15151us) server_negotiation.cc:407] Connection header received
0809 12:08:34.619430 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.619436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.619552 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.619640 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:34.621573 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.622766 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.624906 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.625521 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.625633 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.631692 (+ 6059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:34.631709 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:34.631727 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:34.631767 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:34.635813 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.636743 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.636753 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.636757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.636937 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.637417 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.637422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.637425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.637882 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:34.638193 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.638412 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.638688 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:08:34.639824 27207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:34.731575 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 27033
I20260809 12:08:34.759212 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 27223
Aug 09 12:08:34 dist-test-slave-sb2z krb5kdc[26875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:08:34.777402 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.760166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60140 (local address 127.0.147.66:33443)
0809 12:08:34.760358 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.760362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.760385 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:08:34.760590 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.760594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.760660 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.760744 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:34.762477 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.763255 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.765186 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.765624 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.765720 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.771227 (+ 5507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:34.771242 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:34.771263 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:34.771300 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:34.774503 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.775355 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.775361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.775363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.775505 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.775964 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.775967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.775969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.776314 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:34.776586 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.776902 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.777078 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:08:34.778192 27399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:34.896818 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 27223
I20260809 12:08:34.922416 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 27415
Aug 09 12:08:34 dist-test-slave-sb2z krb5kdc[26875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:08:34.937342 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:34.923140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.0.147.67:33895)
0809 12:08:34.923307 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:34.923311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:34.923328 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:08:34.923490 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:34.923494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:34.923610 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:34.923698 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:34.925097 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.925846 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.927119 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:34.927696 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:34.927791 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:34.932149 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:34.932162 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:34.932178 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:34.932209 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:34.934683 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.935404 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.935411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.935413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.935781 (+ 368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:34.936114 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:34.936117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:34.936119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:34.936453 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:34.936677 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:34.936913 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 12:08:34.937073 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:08:34.938057 27591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:35.029589 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 27415
I20260809 12:08:35.055336 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 26894
I20260809 12:08:35.056468 26987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:35.167919 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 26894
2026-08-09T12:08:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4944 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T12:08:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:35Z Disabled control of system clock
I20260809 12:08:35.269577 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:34423
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:34423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:08:35.613323 27672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:35.613682 27672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:35.613855 27672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:35.621100 27672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:35.621258 27672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:35.621354 27672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:35.621446 27672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:35.621532 27672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:35.630683 27672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:34423
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:34423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:35.632068 27672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:35.633867 27672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:35.646109 27683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:35.646162 27680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:35.646198 27679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:35.646975 27672 server_base.cc:1034] running on GCE node
I20260809 12:08:35.647842 27672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:35.649173 27672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:35.650374 27672 hybrid_clock.cc:648] HybridClock initialized: now 1786277315650323 us; error 50 us; skew 500 ppm
I20260809 12:08:35.650738 27672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:35.653012 27672 webserver.cc:460] Webserver started at http://127.0.147.126:43799/ using document root <none> and password file <none>
I20260809 12:08:35.653707 27672 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:35.653828 27672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:35.654165 27672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:35.657080 27672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "b0d4531881d24b819d9451bc0cd679e5"
format_stamp: "Formatted at 2026-08-09 12:08:35 on dist-test-slave-sb2z"
server_key: "77d4d4f14456a89da935f4be1e2fb4da"
server_key_iv: "557b003dfa51ebd27647ddf2d4c19867"
server_key_version: "encryptionkey@0"
I20260809 12:08:35.657896 27672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "b0d4531881d24b819d9451bc0cd679e5"
format_stamp: "Formatted at 2026-08-09 12:08:35 on dist-test-slave-sb2z"
server_key: "77d4d4f14456a89da935f4be1e2fb4da"
server_key_iv: "557b003dfa51ebd27647ddf2d4c19867"
server_key_version: "encryptionkey@0"
I20260809 12:08:35.662955 27672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:08:35.666332 27690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:35.667711 27672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:08:35.667900 27672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "b0d4531881d24b819d9451bc0cd679e5"
format_stamp: "Formatted at 2026-08-09 12:08:35 on dist-test-slave-sb2z"
server_key: "77d4d4f14456a89da935f4be1e2fb4da"
server_key_iv: "557b003dfa51ebd27647ddf2d4c19867"
server_key_version: "encryptionkey@0"
I20260809 12:08:35.668125 27672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:35.692906 27672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:35.693724 27672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:35.694020 27672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:35.713231 27672 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:34423
I20260809 12:08:35.713251 27771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:34423 every 8 connection(s)
I20260809 12:08:35.714548 27672 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:35.718223 27772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:35.720259 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 27672
I20260809 12:08:35.720448 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:35.720925 589 external_mini_cluster.cc:1428] Setting key 5dfefedb6e7c82b7831fde9434059ef0
I20260809 12:08:35.730019 27772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5: Bootstrap starting.
I20260809 12:08:35.733662 27772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:35.734601 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:35.722853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.0.147.126:34423)
0809 12:08:35.723905 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:35.723916 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:35.723961 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:08:35.725609 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:35.725630 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:35.726059 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:35.726390 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:35.727017 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:35.728397 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:35.730247 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:35.730922 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:35.731052 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:35.732328 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:35.732335 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:35.732347 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:35.732864 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:35.732949 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:35.732969 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:35.733054 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:35.733292 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 12:08:35.733635 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":385,"threads_started":1}
I20260809 12:08:35.734733 27772 log.cc:824] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:35.737704 27772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5: No bootstrap required, opened a new log
I20260809 12:08:35.745177 27772 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4531881d24b819d9451bc0cd679e5" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34423 } }
I20260809 12:08:35.745519 27772 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:35.745676 27772 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0d4531881d24b819d9451bc0cd679e5, State: Initialized, Role: FOLLOWER
I20260809 12:08:35.746282 27772 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4531881d24b819d9451bc0cd679e5" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34423 } }
I20260809 12:08:35.746518 27772 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:35.746677 27772 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:35.746846 27772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:35.749408 27772 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4531881d24b819d9451bc0cd679e5" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34423 } }
I20260809 12:08:35.749922 27772 leader_election.cc:302] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0d4531881d24b819d9451bc0cd679e5; no voters:
I20260809 12:08:35.750538 27772 leader_election.cc:288] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:35.750690 27779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:35.751431 27779 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [term 1 LEADER]: Becoming Leader. State: Replica: b0d4531881d24b819d9451bc0cd679e5, State: Running, Role: LEADER
I20260809 12:08:35.752017 27779 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4531881d24b819d9451bc0cd679e5" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34423 } }
I20260809 12:08:35.752712 27772 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:35.754683 27780 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0d4531881d24b819d9451bc0cd679e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4531881d24b819d9451bc0cd679e5" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34423 } } }
I20260809 12:08:35.755198 27780 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:35.754819 27781 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0d4531881d24b819d9451bc0cd679e5. Latest consensus state: current_term: 1 leader_uuid: "b0d4531881d24b819d9451bc0cd679e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4531881d24b819d9451bc0cd679e5" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34423 } } }
I20260809 12:08:35.759317 27781 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:35.759485 27786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:35.764863 27786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:35.773072 27786 catalog_manager.cc:1347] Generated new cluster ID: 51856a8a23c54a599da92dfbbb290e94
I20260809 12:08:35.773190 27786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:35.782233 27786 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:35.783660 27786 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:35.808501 27786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5: Generated new TSK 0
I20260809 12:08:35.809337 27786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:08:35.822343 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34423
--builtin_ntp_servers=127.0.147.84:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:08:36.168337 27801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:36.168659 27801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:36.168818 27801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:36.176144 27801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:36.176273 27801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:36.176451 27801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:08:36.185314 27801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:36.186642 27801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:36.188422 27801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:36.201923 27809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:36.202014 27813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:36.202055 27808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:36.203351 27801 server_base.cc:1034] running on GCE node
I20260809 12:08:36.203971 27801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:36.204864 27801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:36.206120 27801 hybrid_clock.cc:648] HybridClock initialized: now 1786277316206075 us; error 58 us; skew 500 ppm
I20260809 12:08:36.206429 27801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:36.208675 27801 webserver.cc:460] Webserver started at http://127.0.147.65:38083/ using document root <none> and password file <none>
I20260809 12:08:36.209363 27801 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:36.209498 27801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:36.209836 27801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:36.212738 27801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "f2eb36a204564a2f909e55d03c224ef5"
format_stamp: "Formatted at 2026-08-09 12:08:36 on dist-test-slave-sb2z"
server_key: "5539e4329202e7a76fc6e55484107a98"
server_key_iv: "857ccbbc4f713109a36896a099198abf"
server_key_version: "encryptionkey@0"
I20260809 12:08:36.213527 27801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "f2eb36a204564a2f909e55d03c224ef5"
format_stamp: "Formatted at 2026-08-09 12:08:36 on dist-test-slave-sb2z"
server_key: "5539e4329202e7a76fc6e55484107a98"
server_key_iv: "857ccbbc4f713109a36896a099198abf"
server_key_version: "encryptionkey@0"
I20260809 12:08:36.219233 27801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:08:36.222777 27820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:36.224213 27801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:08:36.224395 27801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "f2eb36a204564a2f909e55d03c224ef5"
format_stamp: "Formatted at 2026-08-09 12:08:36 on dist-test-slave-sb2z"
server_key: "5539e4329202e7a76fc6e55484107a98"
server_key_iv: "857ccbbc4f713109a36896a099198abf"
server_key_version: "encryptionkey@0"
I20260809 12:08:36.224619 27801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:36.248673 27801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:36.249524 27801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:36.249828 27801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:36.250761 27801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:36.252328 27801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:36.252456 27801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:36.252586 27801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:36.252681 27801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:36.286502 27801 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:40705
I20260809 12:08:36.286595 27991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:40705 every 8 connection(s)
I20260809 12:08:36.287865 27801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:08:36.297770 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 27801
I20260809 12:08:36.297946 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:08:36.298496 589 external_mini_cluster.cc:1428] Setting key 7f13ce18b828cd8d45eccf7eae3a50b2
I20260809 12:08:36.300529 27994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:36.290398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:34573 (local address 127.0.147.126:34423)
0809 12:08:36.290864 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:36.290869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:36.291925 (+ 1056us) server_negotiation.cc:407] Connection header received
0809 12:08:36.293741 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:36.293747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:36.293839 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:36.293958 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:36.294987 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:36.295817 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:36.297351 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:36.298037 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:36.298176 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:36.299640 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:36.299645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:36.299648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:36.299664 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:36.299708 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:36.299720 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:36.299788 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:36.300049 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 12:08:36.300158 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":262,"threads_started":1}
I20260809 12:08:36.301438 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34423
--builtin_ntp_servers=127.0.147.84:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:08:36.301542 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:36.290835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:34423 (local address 127.0.147.65:34573)
0809 12:08:36.291775 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:36.291811 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:36.293477 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:36.294200 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:36.294207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:36.294292 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:36.294816 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:36.294831 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:36.295983 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:36.295987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:36.297196 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:36.297208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:36.298257 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:36.298263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:36.298472 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:36.299241 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:08:36.299244 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:36.299287 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:08:36.299365 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:36.299824 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:36.299835 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:36.299843 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:08:36.300030 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 12:08:36.300439 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":258,"threads_started":1}
I20260809 12:08:36.302995 27992 heartbeater.cc:347] Connected to a master server at 127.0.147.126:34423
I20260809 12:08:36.303417 27992 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:36.304335 27992 heartbeater.cc:511] Master 127.0.147.126:34423 requested a full tablet report, sending...
I20260809 12:08:36.306464 27720 ts_manager.cc:194] Registered new tserver with Master: f2eb36a204564a2f909e55d03c224ef5 (127.0.147.65:40705)
I20260809 12:08:36.308046 27720 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.65:34573
W20260809 12:08:36.643428 27996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:36.643777 27996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:36.643939 27996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:36.650944 27996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:36.651032 27996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:36.651235 27996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:08:36.660494 27996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:36.661736 27996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:36.663482 27996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:36.676774 28004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:36.676738 28007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:36.676842 28005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:36.677587 27996 server_base.cc:1034] running on GCE node
I20260809 12:08:36.678236 27996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:36.679153 27996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:36.680388 27996 hybrid_clock.cc:648] HybridClock initialized: now 1786277316680345 us; error 54 us; skew 500 ppm
I20260809 12:08:36.680713 27996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:36.682883 27996 webserver.cc:460] Webserver started at http://127.0.147.66:38763/ using document root <none> and password file <none>
I20260809 12:08:36.683633 27996 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:36.683768 27996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:36.684106 27996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:36.686940 27996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "a749260224bc4cc6afd7c08818abc4c2"
format_stamp: "Formatted at 2026-08-09 12:08:36 on dist-test-slave-sb2z"
server_key: "edbf6e23073c27b17bbe97d4bb06ab7a"
server_key_iv: "a37043ad7f35314135431908ca5e317f"
server_key_version: "encryptionkey@0"
I20260809 12:08:36.687826 27996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "a749260224bc4cc6afd7c08818abc4c2"
format_stamp: "Formatted at 2026-08-09 12:08:36 on dist-test-slave-sb2z"
server_key: "edbf6e23073c27b17bbe97d4bb06ab7a"
server_key_iv: "a37043ad7f35314135431908ca5e317f"
server_key_version: "encryptionkey@0"
I20260809 12:08:36.692869 27996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:08:36.696260 28017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:36.697551 27996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:08:36.697728 27996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "a749260224bc4cc6afd7c08818abc4c2"
format_stamp: "Formatted at 2026-08-09 12:08:36 on dist-test-slave-sb2z"
server_key: "edbf6e23073c27b17bbe97d4bb06ab7a"
server_key_iv: "a37043ad7f35314135431908ca5e317f"
server_key_version: "encryptionkey@0"
I20260809 12:08:36.697950 27996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:36.716174 27996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:36.716975 27996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:36.717293 27996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:36.718267 27996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:36.719843 27996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:36.719967 27996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:36.720117 27996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:36.720211 27996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:36.755303 27996 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:34543
I20260809 12:08:36.755369 28166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:34543 every 8 connection(s)
I20260809 12:08:36.756733 27996 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:08:36.766508 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 27996
I20260809 12:08:36.766734 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:08:36.767352 589 external_mini_cluster.cc:1428] Setting key c79544092d160d9b5194bdfe912c8150
I20260809 12:08:36.769093 27994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:36.759387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:38871 (local address 127.0.147.126:34423)
0809 12:08:36.759636 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:36.759641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:36.760949 (+ 1308us) server_negotiation.cc:407] Connection header received
0809 12:08:36.762666 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:36.762670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:36.762735 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:36.762824 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:36.763909 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:36.764605 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:36.766096 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:36.766490 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:36.766589 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:36.768213 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:36.768218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:36.768222 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:36.768239 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:36.768272 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:36.768283 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:36.768404 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:36.768651 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 12:08:36.768752 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:08:36.770390 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:36.759892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:34423 (local address 127.0.147.66:38871)
0809 12:08:36.760789 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:36.760830 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:36.762427 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:36.763018 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:36.763025 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:36.763107 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:36.763755 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:36.763774 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:36.764735 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:36.764739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:36.765974 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:36.765986 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:36.766735 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:36.766743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:36.767045 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:36.767844 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:08:36.767848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:36.767898 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:08:36.768001 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:36.768405 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:36.768419 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:36.768429 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:08:36.768620 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 12:08:36.769012 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":277,"threads_started":1}
I20260809 12:08:36.770550 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34423
--builtin_ntp_servers=127.0.147.84:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:08:36.771826 28168 heartbeater.cc:347] Connected to a master server at 127.0.147.126:34423
I20260809 12:08:36.772166 28168 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:36.772884 28168 heartbeater.cc:511] Master 127.0.147.126:34423 requested a full tablet report, sending...
I20260809 12:08:36.774281 27720 ts_manager.cc:194] Registered new tserver with Master: a749260224bc4cc6afd7c08818abc4c2 (127.0.147.66:34543)
I20260809 12:08:36.775118 27720 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.66:38871
W20260809 12:08:37.111467 28171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:37.111757 28171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:37.111928 28171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:37.119102 28171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:37.119194 28171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:37.119391 28171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:08:37.128533 28171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:37.129856 28171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:37.131676 28171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:37.144937 28179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:37.145006 28184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:37.144980 28181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:37.146011 28171 server_base.cc:1034] running on GCE node
I20260809 12:08:37.146622 28171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:37.147578 28171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:37.148797 28171 hybrid_clock.cc:648] HybridClock initialized: now 1786277317148742 us; error 51 us; skew 500 ppm
I20260809 12:08:37.149114 28171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:37.154810 28171 webserver.cc:460] Webserver started at http://127.0.147.67:36737/ using document root <none> and password file <none>
I20260809 12:08:37.155478 28171 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:37.155638 28171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:37.155963 28171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:37.158820 28171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "8d79c49a39ed4e188a4cebfaa083f4fa"
format_stamp: "Formatted at 2026-08-09 12:08:37 on dist-test-slave-sb2z"
server_key: "f860a122a422a6d0fa5ef17f2af76d24"
server_key_iv: "7d8cd2bb1a63a2e975ff981186089614"
server_key_version: "encryptionkey@0"
I20260809 12:08:37.159654 28171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "8d79c49a39ed4e188a4cebfaa083f4fa"
format_stamp: "Formatted at 2026-08-09 12:08:37 on dist-test-slave-sb2z"
server_key: "f860a122a422a6d0fa5ef17f2af76d24"
server_key_iv: "7d8cd2bb1a63a2e975ff981186089614"
server_key_version: "encryptionkey@0"
I20260809 12:08:37.164623 28171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:08:37.167846 28192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:37.169133 28171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:08:37.169301 28171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "8d79c49a39ed4e188a4cebfaa083f4fa"
format_stamp: "Formatted at 2026-08-09 12:08:37 on dist-test-slave-sb2z"
server_key: "f860a122a422a6d0fa5ef17f2af76d24"
server_key_iv: "7d8cd2bb1a63a2e975ff981186089614"
server_key_version: "encryptionkey@0"
I20260809 12:08:37.169512 28171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:37.192504 28171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:37.193295 28171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:37.193666 28171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:37.194626 28171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:37.196197 28171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:37.196328 28171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:37.196457 28171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:37.196545 28171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:37.230517 28171 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:43897
I20260809 12:08:37.230609 28361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:43897 every 8 connection(s)
I20260809 12:08:37.231865 28171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:08:37.234586 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 28171
I20260809 12:08:37.234774 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:08:37.235216 589 external_mini_cluster.cc:1428] Setting key d24a8b088e088cfad074db5500dd470e
I20260809 12:08:37.244304 27994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.234486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:58223 (local address 127.0.147.126:34423)
0809 12:08:37.234694 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.234700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.236338 (+ 1638us) server_negotiation.cc:407] Connection header received
0809 12:08:37.238384 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.238388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.238451 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.238546 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:37.239489 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.240280 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.241759 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.242159 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.242253 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.243500 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:37.243512 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:37.243515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:37.243589 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:37.243621 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:37.243634 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:37.243741 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.243997 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.244074 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 12:08:37.245246 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.235102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:34423 (local address 127.0.147.67:58223)
0809 12:08:37.236172 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:37.236226 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:37.238156 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:37.238726 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:37.238733 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:37.238817 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:37.239351 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.239367 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.240434 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.240438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.241630 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.241642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.242345 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.242349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.242536 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.243224 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:08:37.243227 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:37.243262 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:08:37.243336 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:37.243770 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:37.243782 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:37.243790 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:08:37.243994 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 12:08:37.244268 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":380,"threads_started":1}
I20260809 12:08:37.246412 28364 heartbeater.cc:347] Connected to a master server at 127.0.147.126:34423
I20260809 12:08:37.246693 28364 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:37.247398 28364 heartbeater.cc:511] Master 127.0.147.126:34423 requested a full tablet report, sending...
I20260809 12:08:37.248842 27720 ts_manager.cc:194] Registered new tserver with Master: 8d79c49a39ed4e188a4cebfaa083f4fa (127.0.147.67:43897)
I20260809 12:08:37.249893 27720 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.67:58223
I20260809 12:08:37.250341 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:08:37.260787 27994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.254715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.0.147.126:34423)
0809 12:08:37.254886 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.254890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.255389 (+ 499us) server_negotiation.cc:407] Connection header received
0809 12:08:37.255520 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.255565 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.255643 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.255721 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:37.256188 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.256935 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.258355 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.258837 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.258928 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.260087 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:37.260091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:37.260094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:37.260110 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:37.260138 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:37.260148 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:37.260255 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.260416 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.260483 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:08:37.264321 27720 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:08:37.271026 27720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:08:37.295506 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.282789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:34543 (local address 127.0.0.1:49278)
0809 12:08:37.283547 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:37.283591 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:37.283760 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:37.284996 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:37.285003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:37.285039 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:37.285572 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.285591 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.288514 (+ 2923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.288524 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.291666 (+ 3142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.291688 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.294326 (+ 2638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.294336 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.294642 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.294715 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 12:08:37.294833 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 12:08:37.295062 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":318,"threads_started":1}
I20260809 12:08:37.297622 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.282998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.0.147.66:34543)
0809 12:08:37.284327 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.284339 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.284377 (+ 38us) server_negotiation.cc:407] Connection header received
0809 12:08:37.284488 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.284495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.284747 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.285804 (+ 1057us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:37.286221 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.288306 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.292076 (+ 3770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.294059 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.296457 (+ 2398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.296805 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.296888 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.297113 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1136,"thread_start_us":1087,"threads_started":1}
I20260809 12:08:37.302116 28084 tablet_service.cc:1467] Processing CreateTablet for tablet eb5a3b596252463eba1b7c9086000c9d (DEFAULT_TABLE table=test-table [id=be174b8e26a64def93b121ac771595fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:37.303722 28084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb5a3b596252463eba1b7c9086000c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:37.306228 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.284013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.0.1:56150)
0809 12:08:37.284766 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:37.284788 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:37.284933 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:37.299338 (+ 14405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:37.299343 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:37.299372 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:37.299753 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.299765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.301939 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.301945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.304067 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.304079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.305440 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.305446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.305604 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.305645 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 12:08:37.305701 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 12:08:37.305828 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":1,"thread_start_us":398,"threads_started":1}
I20260809 12:08:37.306278 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.285699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.0.147.67:43897)
0809 12:08:37.299348 (+ 13649us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.299357 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.299389 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:08:37.299476 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.299481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.299708 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.299907 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:37.300295 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.301337 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.303740 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.305114 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.305255 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.305518 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.305736 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.305905 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13477,"thread_start_us":13319,"threads_started":1}
I20260809 12:08:37.306226 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.285254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43897 (local address 127.0.0.1:46626)
0809 12:08:37.295822 (+ 10568us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:37.295845 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:37.295961 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:37.299886 (+ 3925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:37.299890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:37.299916 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:37.300133 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.300145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.301502 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.301510 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.303575 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.303590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.305297 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.305305 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.305542 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.305579 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 12:08:37.305675 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 12:08:37.305838 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10426,"mutex_wait_us":29,"thread_start_us":12797,"threads_started":1}
I20260809 12:08:37.307324 28384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.284252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56150 (local address 127.0.147.65:40705)
0809 12:08:37.298750 (+ 14498us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.298759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.298786 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:08:37.298870 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.298877 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.299115 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.299967 (+ 852us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:37.300236 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.301791 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.304256 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.305207 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.306660 (+ 1453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.306923 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.306977 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.307088 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14324,"thread_start_us":14312,"threads_started":1}
I20260809 12:08:37.310202 28271 tablet_service.cc:1467] Processing CreateTablet for tablet eb5a3b596252463eba1b7c9086000c9d (DEFAULT_TABLE table=test-table [id=be174b8e26a64def93b121ac771595fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:37.310745 27891 tablet_service.cc:1467] Processing CreateTablet for tablet eb5a3b596252463eba1b7c9086000c9d (DEFAULT_TABLE table=test-table [id=be174b8e26a64def93b121ac771595fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:37.311499 27992 heartbeater.cc:503] Master 127.0.147.126:34423 was elected leader, sending a full tablet report...
I20260809 12:08:37.311738 28271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb5a3b596252463eba1b7c9086000c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:37.312111 27891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb5a3b596252463eba1b7c9086000c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:37.328456 28388 tablet_bootstrap.cc:492] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: Bootstrap starting.
I20260809 12:08:37.331704 28388 tablet_bootstrap.cc:654] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:37.332603 28391 tablet_bootstrap.cc:492] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: Bootstrap starting.
I20260809 12:08:37.332782 28388 log.cc:824] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:37.333019 28390 tablet_bootstrap.cc:492] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: Bootstrap starting.
I20260809 12:08:37.336247 28391 tablet_bootstrap.cc:654] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:37.336678 28388 tablet_bootstrap.cc:492] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: No bootstrap required, opened a new log
I20260809 12:08:37.336683 28390 tablet_bootstrap.cc:654] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:37.336954 28388 ts_tablet_manager.cc:1397] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 12:08:37.337913 28390 log.cc:824] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:37.338719 28391 log.cc:824] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:37.342188 28391 tablet_bootstrap.cc:492] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: No bootstrap required, opened a new log
I20260809 12:08:37.342187 28390 tablet_bootstrap.cc:492] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: No bootstrap required, opened a new log
I20260809 12:08:37.342454 28390 ts_tablet_manager.cc:1397] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 12:08:37.342448 28391 ts_tablet_manager.cc:1397] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 12:08:37.345005 28388 raft_consensus.cc:348] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } }
I20260809 12:08:37.345350 28388 raft_consensus.cc:374] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:37.345479 28388 raft_consensus.cc:729] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a749260224bc4cc6afd7c08818abc4c2, State: Initialized, Role: FOLLOWER
I20260809 12:08:37.346061 28388 consensus_queue.cc:260] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } }
I20260809 12:08:37.347121 28168 heartbeater.cc:503] Master 127.0.147.126:34423 was elected leader, sending a full tablet report...
I20260809 12:08:37.347824 28388 ts_tablet_manager.cc:1428] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260809 12:08:37.353212 28390 raft_consensus.cc:348] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } }
I20260809 12:08:37.353210 28391 raft_consensus.cc:348] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } }
I20260809 12:08:37.353637 28391 raft_consensus.cc:374] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:37.353636 28390 raft_consensus.cc:374] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:37.353747 28391 raft_consensus.cc:729] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d79c49a39ed4e188a4cebfaa083f4fa, State: Initialized, Role: FOLLOWER
I20260809 12:08:37.353747 28390 raft_consensus.cc:729] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2eb36a204564a2f909e55d03c224ef5, State: Initialized, Role: FOLLOWER
I20260809 12:08:37.354329 28390 consensus_queue.cc:260] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } }
I20260809 12:08:37.354323 28391 consensus_queue.cc:260] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } }
I20260809 12:08:37.355513 28364 heartbeater.cc:503] Master 127.0.147.126:34423 was elected leader, sending a full tablet report...
I20260809 12:08:37.357020 28390 ts_tablet_manager.cc:1428] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: Time spent starting tablet: real 0.014s user 0.016s sys 0.001s
I20260809 12:08:37.357553 28391 ts_tablet_manager.cc:1428] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260809 12:08:37.483654 28365 tablet.cc:2369] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:08:37.508893 28169 tablet.cc:2369] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:08:37.542341 27993 tablet.cc:2369] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:08:37.646090 28397 raft_consensus.cc:482] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:08:37.646379 28397 raft_consensus.cc:504] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } }
I20260809 12:08:37.648237 28397 leader_election.cc:288] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2eb36a204564a2f909e55d03c224ef5 (127.0.147.65:40705), 8d79c49a39ed4e188a4cebfaa083f4fa (127.0.147.67:43897)
I20260809 12:08:37.655253 28384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.648718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:56207 (local address 127.0.147.65:40705)
0809 12:08:37.648946 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.648957 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.649293 (+ 336us) server_negotiation.cc:407] Connection header received
0809 12:08:37.649549 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.649554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.649628 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.649727 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:37.650298 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.650996 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.653199 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.654366 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.654547 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.654630 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.654886 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.655009 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:08:37.655336 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.648430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.147.66:56207)
0809 12:08:37.649159 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:37.649179 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:37.649391 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:37.649753 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:37.649757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:37.649783 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:37.650146 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.650156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.651135 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.651142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.652968 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.652981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.654537 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.654545 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.654724 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.654748 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:08:37.654861 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 12:08:37.655004 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":594,"threads_started":1}
I20260809 12:08:37.655751 28407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.648744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:43897 (local address 127.0.147.66:59797)
0809 12:08:37.649498 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:37.649512 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:37.649670 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:37.650009 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:37.650013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:37.650036 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:37.650403 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.650411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.651393 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.651400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.653088 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:37.653100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.654836 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.654841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:37.655040 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.655075 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:08:37.655254 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 12:08:37.655427 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":422,"threads_started":1}
I20260809 12:08:37.655937 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.648784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:59797 (local address 127.0.147.67:43897)
0809 12:08:37.649003 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.649023 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.649617 (+ 594us) server_negotiation.cc:407] Connection header received
0809 12:08:37.649822 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.649827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.649886 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.649983 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:37.650556 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.651244 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.653314 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.654456 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.654656 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.654722 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.655286 (+ 564us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.655416 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:08:37.656688 27919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb5a3b596252463eba1b7c9086000c9d" candidate_uuid: "a749260224bc4cc6afd7c08818abc4c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2eb36a204564a2f909e55d03c224ef5" is_pre_election: true
I20260809 12:08:37.656895 28297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb5a3b596252463eba1b7c9086000c9d" candidate_uuid: "a749260224bc4cc6afd7c08818abc4c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" is_pre_election: true
I20260809 12:08:37.657285 27919 raft_consensus.cc:2454] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a749260224bc4cc6afd7c08818abc4c2 in term 0.
I20260809 12:08:37.657398 28297 raft_consensus.cc:2454] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a749260224bc4cc6afd7c08818abc4c2 in term 0.
I20260809 12:08:37.658049 28020 leader_election.cc:302] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8d79c49a39ed4e188a4cebfaa083f4fa, a749260224bc4cc6afd7c08818abc4c2, f2eb36a204564a2f909e55d03c224ef5; no voters:
I20260809 12:08:37.658456 28397 raft_consensus.cc:2781] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:08:37.658624 28397 raft_consensus.cc:482] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:08:37.658738 28397 raft_consensus.cc:3037] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:37.661031 28397 raft_consensus.cc:504] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } }
I20260809 12:08:37.661650 28397 leader_election.cc:288] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [CANDIDATE]: Term 1 election: Requested vote from peers f2eb36a204564a2f909e55d03c224ef5 (127.0.147.65:40705), 8d79c49a39ed4e188a4cebfaa083f4fa (127.0.147.67:43897)
I20260809 12:08:37.662011 27919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb5a3b596252463eba1b7c9086000c9d" candidate_uuid: "a749260224bc4cc6afd7c08818abc4c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2eb36a204564a2f909e55d03c224ef5"
I20260809 12:08:37.662154 28297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb5a3b596252463eba1b7c9086000c9d" candidate_uuid: "a749260224bc4cc6afd7c08818abc4c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa"
I20260809 12:08:37.662231 27919 raft_consensus.cc:3037] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:37.662359 28297 raft_consensus.cc:3037] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:37.665530 27919 raft_consensus.cc:2454] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a749260224bc4cc6afd7c08818abc4c2 in term 1.
I20260809 12:08:37.665539 28297 raft_consensus.cc:2454] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a749260224bc4cc6afd7c08818abc4c2 in term 1.
I20260809 12:08:37.665939 28020 leader_election.cc:302] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8d79c49a39ed4e188a4cebfaa083f4fa, a749260224bc4cc6afd7c08818abc4c2, f2eb36a204564a2f909e55d03c224ef5; no voters:
I20260809 12:08:37.666222 28397 raft_consensus.cc:2781] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:37.666640 28397 raft_consensus.cc:686] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 1 LEADER]: Becoming Leader. State: Replica: a749260224bc4cc6afd7c08818abc4c2, State: Running, Role: LEADER
I20260809 12:08:37.667152 28397 consensus_queue.cc:237] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } }
I20260809 12:08:37.671761 27718 catalog_manager.cc:5495] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a749260224bc4cc6afd7c08818abc4c2 (127.0.147.66). New cstate: current_term: 1 leader_uuid: "a749260224bc4cc6afd7c08818abc4c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a749260224bc4cc6afd7c08818abc4c2" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 } health_report { overall_health: UNKNOWN } } }
I20260809 12:08:37.755458 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.749446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49292 (local address 127.0.147.66:34543)
0809 12:08:37.749664 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.749672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.749793 (+ 121us) server_negotiation.cc:407] Connection header received
0809 12:08:37.750070 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.750074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.750165 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.750328 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:08:37.750754 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.751457 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.753064 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.753481 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.753620 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.754227 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:08:37.754714 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:08:37.754951 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.755078 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.755125 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:08:37.765002 27919 raft_consensus.cc:1261] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [term 1 FOLLOWER]: Refusing update from remote peer a749260224bc4cc6afd7c08818abc4c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:37.765002 28297 raft_consensus.cc:1261] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [term 1 FOLLOWER]: Refusing update from remote peer a749260224bc4cc6afd7c08818abc4c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:37.766045 28408 consensus_queue.cc:1035] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d79c49a39ed4e188a4cebfaa083f4fa" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 43897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:37.766407 28397 consensus_queue.cc:1035] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2eb36a204564a2f909e55d03c224ef5" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:37.777462 28414 mvcc.cc:205] Tried to move back new op lower bound from 7316591893554642944 to 7316591893168238592. Current Snapshot: MvccSnapshot[applied={T|T < 7316591893554642944}]
I20260809 12:08:37.790899 27720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:37.791177 27720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:37.794560 27720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0d4531881d24b819d9451bc0cd679e5: Sending DeleteTablet for 3 replicas of tablet eb5a3b596252463eba1b7c9086000c9d
I20260809 12:08:37.795814 27891 tablet_service.cc:1514] Processing DeleteTablet for tablet eb5a3b596252463eba1b7c9086000c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:37 UTC) from {username='slave'} at 127.0.0.1:56150
I20260809 12:08:37.795843 28084 tablet_service.cc:1514] Processing DeleteTablet for tablet eb5a3b596252463eba1b7c9086000c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:37 UTC) from {username='slave'} at 127.0.0.1:49278
I20260809 12:08:37.796599 28271 tablet_service.cc:1514] Processing DeleteTablet for tablet eb5a3b596252463eba1b7c9086000c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:37 UTC) from {username='slave'} at 127.0.0.1:46626
I20260809 12:08:37.797055 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 27801
I20260809 12:08:37.808154 28435 tablet_replica.cc:330] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: stopping tablet replica
I20260809 12:08:37.809389 28435 raft_consensus.cc:2229] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:37.810711 28435 raft_consensus.cc:2258] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:37.814463 28434 tablet_replica.cc:330] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: stopping tablet replica
I20260809 12:08:37.814893 28434 raft_consensus.cc:2229] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:08:37.814513 28435 ts_tablet_manager.cc:1905] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:37.815614 28434 raft_consensus.cc:2258] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:37.816365 28433 tablet_replica.cc:330] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: stopping tablet replica
I20260809 12:08:37.816790 28433 raft_consensus.cc:2229] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:37.817306 28433 raft_consensus.cc:2258] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:37.819427 28433 ts_tablet_manager.cc:1905] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:37.822767 28435 ts_tablet_manager.cc:1918] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:37.822994 28435 log.cc:1196] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/eb5a3b596252463eba1b7c9086000c9d
I20260809 12:08:37.823661 28435 ts_tablet_manager.cc:1939] T eb5a3b596252463eba1b7c9086000c9d P 8d79c49a39ed4e188a4cebfaa083f4fa: Deleting consensus metadata
I20260809 12:08:37.823891 28384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.798025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56160 (local address 127.0.147.65:40705)
0809 12:08:37.798240 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.798249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.815169 (+ 16920us) server_negotiation.cc:407] Connection header received
0809 12:08:37.815503 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.815508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.815652 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.815743 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:37.816476 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.817719 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.819779 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.820441 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.820546 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.822086 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:37.822099 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:37.822117 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:37.822832 (+ 715us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:37.822895 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:37.822932 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:37.823029 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.823329 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.823436 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:08:37.825192 27986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:37.826134 27696 catalog_manager.cc:4841] TS 8d79c49a39ed4e188a4cebfaa083f4fa (127.0.147.67:43897): tablet eb5a3b596252463eba1b7c9086000c9d (table test-table [id=be174b8e26a64def93b121ac771595fb]) successfully deleted
I20260809 12:08:37.829140 28434 ts_tablet_manager.cc:1905] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:37.834995 28434 ts_tablet_manager.cc:1918] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:37.835199 28434 log.cc:1196] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/eb5a3b596252463eba1b7c9086000c9d
I20260809 12:08:37.835788 28434 ts_tablet_manager.cc:1939] T eb5a3b596252463eba1b7c9086000c9d P a749260224bc4cc6afd7c08818abc4c2: Deleting consensus metadata
I20260809 12:08:37.837309 27696 catalog_manager.cc:4841] TS a749260224bc4cc6afd7c08818abc4c2 (127.0.147.66:34543): tablet eb5a3b596252463eba1b7c9086000c9d (table test-table [id=be174b8e26a64def93b121ac771595fb]) successfully deleted
I20260809 12:08:37.929054 28433 ts_tablet_manager.cc:1918] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:37.929338 28433 log.cc:1196] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/eb5a3b596252463eba1b7c9086000c9d
I20260809 12:08:37.929939 28433 ts_tablet_manager.cc:1939] T eb5a3b596252463eba1b7c9086000c9d P f2eb36a204564a2f909e55d03c224ef5: Deleting consensus metadata
I20260809 12:08:37.929953 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 27801
I20260809 12:08:37.957909 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 27996
W20260809 12:08:37.957937 27694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.147.65:40705 (error 108)
I20260809 12:08:37.960691 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.959902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.0.1:56164)
0809 12:08:37.960325 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:37.960458 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"threads_started":1}
W20260809 12:08:37.961072 27694 catalog_manager.cc:4584] TS f2eb36a204564a2f909e55d03c224ef5 (127.0.147.65:40705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb5a3b596252463eba1b7c9086000c9d: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40705: connect: Connection refused (error 111)
I20260809 12:08:37.970271 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:37.959357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.0.147.66:34543)
0809 12:08:37.960201 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:37.960208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:37.960228 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:08:37.960303 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:37.960310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:37.960413 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:37.960544 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:37.961096 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.962317 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.963823 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:37.964517 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:37.964658 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:37.968547 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:37.968558 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:37.968577 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:37.969417 (+ 840us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:37.969479 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:37.969505 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:37.969613 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:37.969745 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 12:08:37.969857 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694}
I20260809 12:08:37.971113 28162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:38.014779 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:38.014264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.0.1:56166)
0809 12:08:38.014501 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:38.014608 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 12:08:38.015954 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:38.015592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.0.1:56178)
0809 12:08:38.015725 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:38.015813 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 12:08:38.076779 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:38.076313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.0.1:56188)
0809 12:08:38.076528 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:38.076629 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 12:08:38.077900 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:38.077538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.0.1:56204)
0809 12:08:38.077678 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:38.077750 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 12:08:38.088943 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 27996
I20260809 12:08:38.116426 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 28171
I20260809 12:08:38.123456 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:38.117228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46634 (local address 127.0.147.67:43897)
0809 12:08:38.117416 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:38.117421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:38.117440 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:08:38.117598 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:38.117602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:38.117689 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:38.117771 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:38.118154 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:38.118862 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:38.120385 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:38.120800 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:38.120894 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:38.122115 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:38.122123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:38.122137 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:38.122692 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:38.122737 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:38.122762 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:38.122856 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:38.123005 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 12:08:38.123091 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:08:38.124253 28356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:38.158427 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:38.157926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.0.1:56218)
0809 12:08:38.158123 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:38.158245 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 12:08:38.159720 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:38.159230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.0.1:56220)
0809 12:08:38.159434 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:38.159502 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 12:08:38.213233 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 28171
I20260809 12:08:38.238577 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 27672
I20260809 12:08:38.239858 27767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:38.352517 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:38.348295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40705 (local address 127.0.0.1:56228)
0809 12:08:38.352150 (+ 3855us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:38.352296 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3484}
I20260809 12:08:38.353152 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 27672
2026-08-09T12:08:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3187 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:08:38 dist-test-slave-sb2z krb5kdc[28448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:08:38 dist-test-slave-sb2z krb5kdc[28448](info): set up 2 sockets
Aug 09 12:08:38 dist-test-slave-sb2z krb5kdc[28448](info): commencing operation
krb5kdc: starting...
W20260809 12:08:39.677978 589 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:08:39 dist-test-slave-sb2z krb5kdc[28448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:08:40.016975 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45705
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:45705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:40.366765 28467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:40.367112 28467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:40.367295 28467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:40.374575 28467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:40.374717 28467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:40.374841 28467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:40.374950 28467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:40.375034 28467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:40.384711 28467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:45705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:40.385990 28467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:40.387295 28467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:40.399242 28476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:40.399338 28475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:40.399505 28478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:40.400246 28467 server_base.cc:1034] running on GCE node
I20260809 12:08:40.401017 28467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:40.402372 28467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:40.403646 28467 hybrid_clock.cc:648] HybridClock initialized: now 1786277320403591 us; error 70 us; skew 500 ppm
I20260809 12:08:40.404016 28467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:40 dist-test-slave-sb2z krb5kdc[28448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277320, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:40.407058 28467 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:08:40.408581 28467 webserver.cc:460] Webserver started at http://127.0.147.126:36777/ using document root <none> and password file <none>
I20260809 12:08:40.409245 28467 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:40.409397 28467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:40.409721 28467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:40.412704 28467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "a254217baf1f41dcbad129a8f2e657a3"
format_stamp: "Formatted at 2026-08-09 12:08:40 on dist-test-slave-sb2z"
server_key: "e6b1923b998928212a4cfe60024f45b0"
server_key_iv: "49d4ad712c3bd05ac4bc1fd5355171d3"
server_key_version: "encryptionkey@0"
I20260809 12:08:40.413491 28467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "a254217baf1f41dcbad129a8f2e657a3"
format_stamp: "Formatted at 2026-08-09 12:08:40 on dist-test-slave-sb2z"
server_key: "e6b1923b998928212a4cfe60024f45b0"
server_key_iv: "49d4ad712c3bd05ac4bc1fd5355171d3"
server_key_version: "encryptionkey@0"
I20260809 12:08:40.418483 28467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:08:40.421921 28487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:40.423170 28467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:08:40.423364 28467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "a254217baf1f41dcbad129a8f2e657a3"
format_stamp: "Formatted at 2026-08-09 12:08:40 on dist-test-slave-sb2z"
server_key: "e6b1923b998928212a4cfe60024f45b0"
server_key_iv: "49d4ad712c3bd05ac4bc1fd5355171d3"
server_key_version: "encryptionkey@0"
I20260809 12:08:40.423630 28467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:40.442656 28467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:40.446595 28467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:40.446892 28467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:40.466020 28467 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:45705
I20260809 12:08:40.466106 28560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:45705 every 8 connection(s)
I20260809 12:08:40.467417 28467 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:40.471158 28562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:40.472833 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 28467
I20260809 12:08:40.473030 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:40.473483 589 external_mini_cluster.cc:1428] Setting key cc9bb811b3a3020b0066d44a28656f9a
I20260809 12:08:40.484943 28562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3: Bootstrap starting.
Aug 09 12:08:40 dist-test-slave-sb2z krb5kdc[28448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:40.488951 28562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:40.489957 28562 log.cc:824] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:40.492843 28562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3: No bootstrap required, opened a new log
I20260809 12:08:40.497282 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:40.475392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50202 (local address 127.0.147.126:45705)
0809 12:08:40.476308 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:40.476320 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:40.476363 (+ 43us) server_negotiation.cc:407] Connection header received
0809 12:08:40.477992 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:40.478026 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:40.478402 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:40.478704 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:40.480671 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:40.482001 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:40.483874 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:40.484522 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:40.484646 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:40.489764 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:40.489771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:40.489785 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:40.489829 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:40.493081 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:40.493810 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:40.493817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:40.493823 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:40.494004 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:40.494434 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:40.494438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:40.494440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:40.494992 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:40.495334 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:40.495853 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 12:08:40.496284 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":415,"threads_started":1}
I20260809 12:08:40.500896 28562 raft_consensus.cc:348] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a254217baf1f41dcbad129a8f2e657a3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45705 } }
I20260809 12:08:40.501286 28562 raft_consensus.cc:374] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:40.501394 28562 raft_consensus.cc:729] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a254217baf1f41dcbad129a8f2e657a3, State: Initialized, Role: FOLLOWER
I20260809 12:08:40.501979 28562 consensus_queue.cc:260] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a254217baf1f41dcbad129a8f2e657a3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45705 } }
I20260809 12:08:40.502235 28562 raft_consensus.cc:388] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:40.502394 28562 raft_consensus.cc:482] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:40.502573 28562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:40.505203 28562 raft_consensus.cc:504] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a254217baf1f41dcbad129a8f2e657a3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45705 } }
I20260809 12:08:40.505764 28562 leader_election.cc:302] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a254217baf1f41dcbad129a8f2e657a3; no voters:
I20260809 12:08:40.506290 28562 leader_election.cc:288] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:40.506444 28573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:40.507201 28573 raft_consensus.cc:686] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [term 1 LEADER]: Becoming Leader. State: Replica: a254217baf1f41dcbad129a8f2e657a3, State: Running, Role: LEADER
I20260809 12:08:40.507846 28573 consensus_queue.cc:237] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a254217baf1f41dcbad129a8f2e657a3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45705 } }
I20260809 12:08:40.508522 28562 sys_catalog.cc:564] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:40.510936 28574 sys_catalog.cc:455] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a254217baf1f41dcbad129a8f2e657a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a254217baf1f41dcbad129a8f2e657a3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45705 } } }
I20260809 12:08:40.511322 28574 sys_catalog.cc:458] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:40.515084 28578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:40.514971 28575 sys_catalog.cc:455] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a254217baf1f41dcbad129a8f2e657a3. Latest consensus state: current_term: 1 leader_uuid: "a254217baf1f41dcbad129a8f2e657a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a254217baf1f41dcbad129a8f2e657a3" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45705 } } }
I20260809 12:08:40.515324 28575 sys_catalog.cc:458] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:40.521234 28578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:40.531368 28578 catalog_manager.cc:1347] Generated new cluster ID: 1337764530244c3182f39eda3e61419c
I20260809 12:08:40.531571 28578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:40.547441 28578 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:40.548965 28578 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:40.558106 28578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3: Generated new TSK 0
I20260809 12:08:40.558939 28578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.147.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
Entry for principal kudu/127.0.147.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab.
I20260809 12:08:40.654838 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45705
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:41.023401 28609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:41.023739 28609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:41.023852 28609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:41.031150 28609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:41.031255 28609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:41.031366 28609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:08:41.041044 28609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.65.keytab
--principal=kudu/127.0.147.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:41.042260 28609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:41.043694 28609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:41.056520 28615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:41.057467 28617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:41.058626 28619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:41.058848 28609 server_base.cc:1034] running on GCE node
I20260809 12:08:41.059454 28609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:41.060473 28609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:41.061710 28609 hybrid_clock.cc:648] HybridClock initialized: now 1786277321061658 us; error 50 us; skew 500 ppm
I20260809 12:08:41.062012 28609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:41 dist-test-slave-sb2z krb5kdc[28448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:41.065436 28609 init.cc:377] Logged in from keytab as kudu/127.0.147.65@KRBTEST.COM (short username kudu)
I20260809 12:08:41.066859 28609 webserver.cc:460] Webserver started at http://127.0.147.65:41751/ using document root <none> and password file <none>
I20260809 12:08:41.067620 28609 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:41.067768 28609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:41.068085 28609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:41.071044 28609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "d868bc392e9e47359fbc6137c094dbd7"
format_stamp: "Formatted at 2026-08-09 12:08:41 on dist-test-slave-sb2z"
server_key: "2afa415e36b85286d8c47db2afbfa81a"
server_key_iv: "cb8564ae2e0af068d8435279076fd7ff"
server_key_version: "encryptionkey@0"
I20260809 12:08:41.071895 28609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "d868bc392e9e47359fbc6137c094dbd7"
format_stamp: "Formatted at 2026-08-09 12:08:41 on dist-test-slave-sb2z"
server_key: "2afa415e36b85286d8c47db2afbfa81a"
server_key_iv: "cb8564ae2e0af068d8435279076fd7ff"
server_key_version: "encryptionkey@0"
I20260809 12:08:41.077010 28609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 12:08:41.080494 28626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:41.081862 28609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:08:41.082049 28609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "d868bc392e9e47359fbc6137c094dbd7"
format_stamp: "Formatted at 2026-08-09 12:08:41 on dist-test-slave-sb2z"
server_key: "2afa415e36b85286d8c47db2afbfa81a"
server_key_iv: "cb8564ae2e0af068d8435279076fd7ff"
server_key_version: "encryptionkey@0"
I20260809 12:08:41.082274 28609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:41.112056 28609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:41.116096 28609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:41.116421 28609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:41.117462 28609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:41.118782 28609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:41.118889 28609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:41.119040 28609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:41.119141 28609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:41.153082 28609 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:46523
I20260809 12:08:41.153177 28780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:46523 every 8 connection(s)
I20260809 12:08:41.154390 28609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:08:41.155694 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 28609
I20260809 12:08:41.155876 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:08:41.156440 589 external_mini_cluster.cc:1428] Setting key 00d06b741c9278acf2ee579885958230
Aug 09 12:08:41 dist-test-slave-sb2z krb5kdc[28448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.65@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:41.174894 28785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:41.157304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:37093 (local address 127.0.147.126:45705)
0809 12:08:41.157837 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:41.157842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:41.158889 (+ 1047us) server_negotiation.cc:407] Connection header received
0809 12:08:41.160839 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:41.160847 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:41.160952 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:41.161076 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:41.162580 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:41.163341 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:41.164934 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:41.165341 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:41.165484 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:41.169214 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:41.169219 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:41.169223 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:41.169254 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:41.171864 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:41.172573 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:41.172578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:41.172580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:41.172714 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:41.173166 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:41.173171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:41.173173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:41.173433 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:41.173608 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:41.174306 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 12:08:41.174531 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":257,"threads_started":1}
I20260809 12:08:41.176124 28786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:41.157897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45705 (local address 127.0.147.65:37093)
0809 12:08:41.158708 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:41.158755 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:41.160572 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:41.161277 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:41.161284 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:41.161776 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:41.162369 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:41.162398 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:41.163516 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:41.163520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:41.164782 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:41.164794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:41.165575 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:41.165584 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:41.165842 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:41.166548 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:08:41.166571 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:41.168988 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:41.172011 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:41.172019 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:41.172034 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:41.172435 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:41.172829 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:41.172835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:41.172838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:41.173028 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:41.173564 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:41.173572 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:41.173925 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 12:08:41.174181 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 12:08:41.174679 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":354,"threads_started":1}
I20260809 12:08:41.177675 28781 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45705
I20260809 12:08:41.177974 28781 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:41.178675 28781 heartbeater.cc:511] Master 127.0.147.126:45705 requested a full tablet report, sending...
I20260809 12:08:41.180831 28510 ts_manager.cc:194] Registered new tserver with Master: d868bc392e9e47359fbc6137c094dbd7 (127.0.147.65:46523)
I20260809 12:08:41.182320 28510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.65@KRBTEST.COM'} at 127.0.147.65:37093
WARNING: no policy specified for kudu/127.0.147.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
Entry for principal kudu/127.0.147.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab.
I20260809 12:08:41.234831 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45705
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:41.595607 28789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:41.596006 28789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:41.596189 28789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:41.604494 28789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:41.604665 28789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:41.604854 28789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:08:41.615805 28789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.66.keytab
--principal=kudu/127.0.147.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:41.617203 28789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:41.618583 28789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:41.632123 28797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:41.632247 28799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:41.632890 28803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:41.633374 28789 server_base.cc:1034] running on GCE node
I20260809 12:08:41.633996 28789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:41.635152 28789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:41.636359 28789 hybrid_clock.cc:648] HybridClock initialized: now 1786277321636313 us; error 44 us; skew 500 ppm
I20260809 12:08:41.636694 28789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:41 dist-test-slave-sb2z krb5kdc[28448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:41.639905 28789 init.cc:377] Logged in from keytab as kudu/127.0.147.66@KRBTEST.COM (short username kudu)
I20260809 12:08:41.641458 28789 webserver.cc:460] Webserver started at http://127.0.147.66:46271/ using document root <none> and password file <none>
I20260809 12:08:41.642163 28789 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:41.642313 28789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:41.642627 28789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:41.645645 28789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "ade8ce333ea14a3b8533166f1b8c434c"
format_stamp: "Formatted at 2026-08-09 12:08:41 on dist-test-slave-sb2z"
server_key: "938f63ab54415b645ac6d4c6bc51f8af"
server_key_iv: "d3f0ff062b025219e7e3be3e1133c1e5"
server_key_version: "encryptionkey@0"
I20260809 12:08:41.646448 28789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "ade8ce333ea14a3b8533166f1b8c434c"
format_stamp: "Formatted at 2026-08-09 12:08:41 on dist-test-slave-sb2z"
server_key: "938f63ab54415b645ac6d4c6bc51f8af"
server_key_iv: "d3f0ff062b025219e7e3be3e1133c1e5"
server_key_version: "encryptionkey@0"
I20260809 12:08:41.651599 28789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:08:41.654922 28812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:41.656325 28789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:08:41.656517 28789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "ade8ce333ea14a3b8533166f1b8c434c"
format_stamp: "Formatted at 2026-08-09 12:08:41 on dist-test-slave-sb2z"
server_key: "938f63ab54415b645ac6d4c6bc51f8af"
server_key_iv: "d3f0ff062b025219e7e3be3e1133c1e5"
server_key_version: "encryptionkey@0"
I20260809 12:08:41.656716 28789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:41.684659 28789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:41.688856 28789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:41.689172 28789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:41.690150 28789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:41.691512 28789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:41.691685 28789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:41.691841 28789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:41.691916 28789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:41.726306 28789 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:34911
I20260809 12:08:41.726356 28976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:34911 every 8 connection(s)
I20260809 12:08:41.727715 28789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:08:41.729131 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 28789
I20260809 12:08:41.729347 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:08:41.729840 589 external_mini_cluster.cc:1428] Setting key b9a549817e6b714e70ecfeec967bd285
Aug 09 12:08:41 dist-test-slave-sb2z krb5kdc[28448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.66@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:41.747725 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:41.730232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:55639 (local address 127.0.147.126:45705)
0809 12:08:41.731115 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:41.731121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:41.732127 (+ 1006us) server_negotiation.cc:407] Connection header received
0809 12:08:41.733811 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:41.733817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:41.733904 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:41.734041 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:41.735705 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:41.736457 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:41.738017 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:41.738475 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:41.738617 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:41.742053 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:41.742064 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:41.742068 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:41.742099 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:41.744680 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:41.745391 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:41.745402 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:41.745404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:41.745534 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:41.745968 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:41.745974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:41.745979 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:41.746247 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:41.746420 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:41.747095 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 12:08:41.747329 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":625,"threads_started":1}
I20260809 12:08:41.748631 28982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:41.730770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45705 (local address 127.0.147.66:55639)
0809 12:08:41.731694 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:41.731747 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:41.733532 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:41.734285 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:41.734293 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:41.734910 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:41.735479 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:41.735497 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:41.736636 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:41.736640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:41.737854 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:41.737870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:41.738713 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:41.738721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:41.738950 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:41.739675 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:08:41.739697 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:41.741834 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:41.744820 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:41.744828 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:41.744843 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:41.745261 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:41.745650 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:41.745655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:41.745659 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:41.745843 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:41.746381 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:41.746388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:41.746729 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 12:08:41.746985 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 12:08:41.747516 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":396,"threads_started":1}
I20260809 12:08:41.750180 28977 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45705
I20260809 12:08:41.750485 28977 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:41.751190 28977 heartbeater.cc:511] Master 127.0.147.126:45705 requested a full tablet report, sending...
I20260809 12:08:41.752708 28510 ts_manager.cc:194] Registered new tserver with Master: ade8ce333ea14a3b8533166f1b8c434c (127.0.147.66:34911)
I20260809 12:08:41.753535 28510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.66@KRBTEST.COM'} at 127.0.147.66:55639
WARNING: no policy specified for kudu/127.0.147.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
Entry for principal kudu/127.0.147.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab.
I20260809 12:08:41.806313 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45705
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:42.154539 28986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:42.154924 28986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:42.155108 28986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:42.164126 28986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:42.164230 28986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:42.164342 28986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:08:42.174115 28986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/krb5kdc/kudu_127.0.147.67.keytab
--principal=kudu/127.0.147.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:45705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:42.175292 28986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:42.176635 28986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 12:08:42.185223 28781 heartbeater.cc:503] Master 127.0.147.126:45705 was elected leader, sending a full tablet report...
W20260809 12:08:42.189932 28996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:42.189906 28992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:42.189990 28993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:42.190902 28986 server_base.cc:1034] running on GCE node
I20260809 12:08:42.191519 28986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:42.192452 28986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:42.193655 28986 hybrid_clock.cc:648] HybridClock initialized: now 1786277322193599 us; error 48 us; skew 500 ppm
I20260809 12:08:42.193935 28986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:42 dist-test-slave-sb2z krb5kdc[28448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:42.197144 28986 init.cc:377] Logged in from keytab as kudu/127.0.147.67@KRBTEST.COM (short username kudu)
I20260809 12:08:42.198608 28986 webserver.cc:460] Webserver started at http://127.0.147.67:33583/ using document root <none> and password file <none>
I20260809 12:08:42.199263 28986 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:42.199396 28986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:42.199798 28986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:42.202603 28986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "76632b3049d7452eba2a8cf1bd45e545"
format_stamp: "Formatted at 2026-08-09 12:08:42 on dist-test-slave-sb2z"
server_key: "8333e6f7cae8ded781c1c8ca0f757722"
server_key_iv: "389d5dbb0b7d07f995dcb0f89586d85d"
server_key_version: "encryptionkey@0"
I20260809 12:08:42.203375 28986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "76632b3049d7452eba2a8cf1bd45e545"
format_stamp: "Formatted at 2026-08-09 12:08:42 on dist-test-slave-sb2z"
server_key: "8333e6f7cae8ded781c1c8ca0f757722"
server_key_iv: "389d5dbb0b7d07f995dcb0f89586d85d"
server_key_version: "encryptionkey@0"
I20260809 12:08:42.208238 28986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 12:08:42.212306 29007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:42.213526 28986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:08:42.213685 28986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "76632b3049d7452eba2a8cf1bd45e545"
format_stamp: "Formatted at 2026-08-09 12:08:42 on dist-test-slave-sb2z"
server_key: "8333e6f7cae8ded781c1c8ca0f757722"
server_key_iv: "389d5dbb0b7d07f995dcb0f89586d85d"
server_key_version: "encryptionkey@0"
I20260809 12:08:42.213879 28986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:42.246157 28986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:42.250634 28986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:42.250929 28986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:42.251924 28986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:42.253214 28986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:42.253288 28986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:42.253458 28986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:42.253525 28986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:42.287743 28986 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:36945
I20260809 12:08:42.287792 29175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:36945 every 8 connection(s)
I20260809 12:08:42.288995 28986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:08:42.289829 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 28986
I20260809 12:08:42.290027 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:08:42.290459 589 external_mini_cluster.cc:1428] Setting key a919ccdde0c2f4fdabebe2e0255f5d08
Aug 09 12:08:42 dist-test-slave-sb2z krb5kdc[28448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.67@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:08:42.309597 29178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.291658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:59397 (local address 127.0.147.126:45705)
0809 12:08:42.292115 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.292121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.293237 (+ 1116us) server_negotiation.cc:407] Connection header received
0809 12:08:42.295627 (+ 2390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.295632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.295735 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.295852 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:42.297265 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.298022 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.299519 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.299965 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.300105 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.303311 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:42.303316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:42.303319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:42.303350 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:42.306628 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:42.307303 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:42.307309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:42.307316 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:42.307449 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:42.307891 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:42.307897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:42.307900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:42.308157 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:42.308314 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.308961 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.309201 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":199,"threads_started":1}
I20260809 12:08:42.310398 29179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.292340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:45705 (local address 127.0.147.67:59397)
0809 12:08:42.293088 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:42.293132 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:42.295351 (+ 2219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:42.296096 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:42.296103 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:42.296577 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:42.297104 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.297120 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.298179 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.298184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.299379 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.299391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.300203 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.300210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.300429 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.301019 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:08:42.301044 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:42.303105 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:42.306762 (+ 3657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:42.306770 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:42.306784 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:42.307179 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:42.307582 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:08:42.307589 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:08:42.307593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:08:42.307774 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:08:42.308295 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:42.308303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:42.308596 (+ 293us) client_negotiation.cc:769] Sending connection context
0809 12:08:42.308854 (+ 258us) client_negotiation.cc:240] Negotiation successful
0809 12:08:42.309335 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":313,"threads_started":1}
I20260809 12:08:42.311736 29176 heartbeater.cc:347] Connected to a master server at 127.0.147.126:45705
I20260809 12:08:42.312037 29176 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:42.312709 29176 heartbeater.cc:511] Master 127.0.147.126:45705 requested a full tablet report, sending...
I20260809 12:08:42.314105 28510 ts_manager.cc:194] Registered new tserver with Master: 76632b3049d7452eba2a8cf1bd45e545 (127.0.147.67:36945)
I20260809 12:08:42.314864 28510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.147.67@KRBTEST.COM'} at 127.0.147.67:59397
I20260809 12:08:42.318960 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:08:42.344496 29178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.328401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.0.147.126:45705)
0809 12:08:42.328593 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.328602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.328627 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:08:42.328786 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.328794 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.328880 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.328993 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:42.330941 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.332228 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.333635 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.334437 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.334608 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.338085 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:42.338089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:42.338093 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:42.338123 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:42.341259 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:42.341909 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:42.341914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:42.341918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:42.342058 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:42.342469 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:42.342475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:42.342478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:42.342967 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:42.343236 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.343729 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.344115 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 12:08:42.350311 28510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:08:42.357223 28510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:08:42.382567 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.369486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:36945 (local address 127.0.0.1:47158)
0809 12:08:42.370441 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:42.370493 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:42.370733 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:42.372241 (+ 1508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:42.372248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:42.372280 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:42.372670 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.372689 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.375363 (+ 2674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.375372 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.378604 (+ 3232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.378626 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.381187 (+ 2561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.381197 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.381478 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.381555 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 12:08:42.381709 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 12:08:42.381959 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":2069,"threads_started":1}
I20260809 12:08:42.385277 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.369681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.0.147.67:36945)
0809 12:08:42.371559 (+ 1878us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.371571 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.371612 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:08:42.371726 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.371734 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.372000 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.372232 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:42.372905 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.375147 (+ 2242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.378893 (+ 3746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.380916 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.384432 (+ 3516us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.384717 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.384797 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.384955 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1662,"thread_start_us":1467,"threads_started":1}
I20260809 12:08:42.390324 29084 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7c93060ef24db1ac993f5f1813e859 (DEFAULT_TABLE table=test-table [id=6b96d0b9249440069c48c5e4f21484bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:42.391834 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.370288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39742 (local address 127.0.147.65:46523)
0809 12:08:42.383208 (+ 12920us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.383220 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.383259 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:08:42.383391 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.383402 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.383718 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.385061 (+ 1343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:42.385381 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.387106 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.389698 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.390787 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.390944 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.391185 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.391331 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.391480 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12667,"thread_start_us":13621,"threads_started":1}
I20260809 12:08:42.391845 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.369553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46523 (local address 127.0.0.1:39742)
0809 12:08:42.371819 (+ 2266us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:42.371836 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:42.371944 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:42.384265 (+ 12321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:42.384273 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:42.384313 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:42.384781 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.384795 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.387236 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.387243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.389521 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.389536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.390985 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.390992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.391169 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.391203 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:08:42.391292 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 12:08:42.391413 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2156,"mutex_wait_us":3,"thread_start_us":1000,"threads_started":1}
I20260809 12:08:42.392750 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.372780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:34911 (local address 127.0.0.1:50954)
0809 12:08:42.382883 (+ 10103us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:42.382906 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:42.383028 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:42.385803 (+ 2775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:42.385810 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:42.385845 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:42.386125 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.386135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.388095 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.388100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.389973 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.389988 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.391923 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.391932 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.392130 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.392168 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:08:42.392238 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 12:08:42.392375 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9974,"mutex_wait_us":33,"thread_start_us":13084,"threads_started":1}
I20260809 12:08:42.394131 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.372840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50954 (local address 127.0.147.66:34911)
0809 12:08:42.385010 (+ 12170us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.385018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.385046 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:08:42.385155 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.385162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.385389 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.386309 (+ 920us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:42.386588 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.387944 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.390145 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.391726 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.393362 (+ 1636us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.393562 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.393638 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.393823 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12006,"thread_start_us":12683,"threads_started":1}
I20260809 12:08:42.395679 29084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7c93060ef24db1ac993f5f1813e859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:42.395226 28688 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7c93060ef24db1ac993f5f1813e859 (DEFAULT_TABLE table=test-table [id=6b96d0b9249440069c48c5e4f21484bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:42.396631 28688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7c93060ef24db1ac993f5f1813e859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:42.397334 28882 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7c93060ef24db1ac993f5f1813e859 (DEFAULT_TABLE table=test-table [id=6b96d0b9249440069c48c5e4f21484bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:42.398662 28882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7c93060ef24db1ac993f5f1813e859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:42.407693 29212 tablet_bootstrap.cc:492] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: Bootstrap starting.
I20260809 12:08:42.410924 29214 tablet_bootstrap.cc:492] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: Bootstrap starting.
I20260809 12:08:42.411834 29212 tablet_bootstrap.cc:654] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:42.413018 29215 tablet_bootstrap.cc:492] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: Bootstrap starting.
I20260809 12:08:42.413149 29212 log.cc:824] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:42.414712 29214 tablet_bootstrap.cc:654] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:42.415933 29214 log.cc:824] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:42.416196 29215 tablet_bootstrap.cc:654] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:42.417415 29215 log.cc:824] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:42.417896 29212 tablet_bootstrap.cc:492] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: No bootstrap required, opened a new log
I20260809 12:08:42.418228 29212 ts_tablet_manager.cc:1397] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 12:08:42.420398 29215 tablet_bootstrap.cc:492] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: No bootstrap required, opened a new log
I20260809 12:08:42.420646 29215 ts_tablet_manager.cc:1397] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260809 12:08:42.422154 29214 tablet_bootstrap.cc:492] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: No bootstrap required, opened a new log
I20260809 12:08:42.422394 29214 ts_tablet_manager.cc:1397] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260809 12:08:42.428562 29215 raft_consensus.cc:348] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } }
I20260809 12:08:42.428903 29215 raft_consensus.cc:374] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:42.429028 29215 raft_consensus.cc:729] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade8ce333ea14a3b8533166f1b8c434c, State: Initialized, Role: FOLLOWER
I20260809 12:08:42.428953 29212 raft_consensus.cc:348] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } }
I20260809 12:08:42.429363 29212 raft_consensus.cc:374] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:42.429461 29212 raft_consensus.cc:729] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d868bc392e9e47359fbc6137c094dbd7, State: Initialized, Role: FOLLOWER
I20260809 12:08:42.429775 29215 consensus_queue.cc:260] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } }
I20260809 12:08:42.430034 29212 consensus_queue.cc:260] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } }
I20260809 12:08:42.430938 28977 heartbeater.cc:503] Master 127.0.147.126:45705 was elected leader, sending a full tablet report...
I20260809 12:08:42.432482 29215 ts_tablet_manager.cc:1428] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260809 12:08:42.434355 29214 raft_consensus.cc:348] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } }
I20260809 12:08:42.434864 29214 raft_consensus.cc:374] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:42.434989 29214 raft_consensus.cc:729] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76632b3049d7452eba2a8cf1bd45e545, State: Initialized, Role: FOLLOWER
I20260809 12:08:42.435699 29214 consensus_queue.cc:260] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } }
I20260809 12:08:42.436897 29176 heartbeater.cc:503] Master 127.0.147.126:45705 was elected leader, sending a full tablet report...
I20260809 12:08:42.438241 29212 ts_tablet_manager.cc:1428] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: Time spent starting tablet: real 0.020s user 0.007s sys 0.013s
I20260809 12:08:42.438704 29214 ts_tablet_manager.cc:1428] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
W20260809 12:08:42.479815 28979 tablet.cc:2369] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:08:42.540729 29177 tablet.cc:2369] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:08:42.657195 28783 tablet.cc:2369] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:08:42.661458 29222 raft_consensus.cc:482] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:08:42.661737 29222 raft_consensus.cc:504] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } }
I20260809 12:08:42.663432 29222 leader_election.cc:288] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76632b3049d7452eba2a8cf1bd45e545 (127.0.147.67:36945), d868bc392e9e47359fbc6137c094dbd7 (127.0.147.65:46523)
I20260809 12:08:42.672793 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.663903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:56107 (local address 127.0.147.67:36945)
0809 12:08:42.664143 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.664153 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.664601 (+ 448us) server_negotiation.cc:407] Connection header received
0809 12:08:42.664827 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.664833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.664922 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.665019 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:42.665857 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.667039 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.670218 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.671476 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.671691 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.671753 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.672404 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.672535 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:08:42.672793 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.663905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:60855 (local address 127.0.147.65:46523)
0809 12:08:42.664133 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.664141 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.664828 (+ 687us) server_negotiation.cc:407] Connection header received
0809 12:08:42.664904 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.664909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.664990 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.665089 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:42.665914 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.667025 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.670213 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.671464 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.671678 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.671754 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.672404 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.672535 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:08:42.673051 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.663731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:36945 (local address 127.0.147.66:56107)
0809 12:08:42.664465 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:42.664482 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:42.664668 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:42.665067 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:42.665073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:42.665101 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:42.665671 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.665689 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.667201 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.667206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.670050 (+ 2844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.670066 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.671773 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.671780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.672123 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.672259 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 12:08:42.672359 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 12:08:42.672546 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":298,"thread_start_us":432,"threads_started":1}
I20260809 12:08:42.673074 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.663780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:46523 (local address 127.0.147.66:60855)
0809 12:08:42.664659 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:42.664678 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:42.664790 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:42.665101 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:42.665105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:42.665122 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:42.665727 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.665740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.667202 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.667206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.670050 (+ 2844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:42.670066 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.671773 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.671780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:42.672124 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.672258 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 12:08:42.672359 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 12:08:42.672543 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":384,"thread_start_us":570,"threads_started":1}
I20260809 12:08:42.674190 29115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7c93060ef24db1ac993f5f1813e859" candidate_uuid: "ade8ce333ea14a3b8533166f1b8c434c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76632b3049d7452eba2a8cf1bd45e545" is_pre_election: true
I20260809 12:08:42.674191 28717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7c93060ef24db1ac993f5f1813e859" candidate_uuid: "ade8ce333ea14a3b8533166f1b8c434c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d868bc392e9e47359fbc6137c094dbd7" is_pre_election: true
I20260809 12:08:42.674783 29115 raft_consensus.cc:2454] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade8ce333ea14a3b8533166f1b8c434c in term 0.
I20260809 12:08:42.674784 28717 raft_consensus.cc:2454] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade8ce333ea14a3b8533166f1b8c434c in term 0.
I20260809 12:08:42.675559 28814 leader_election.cc:302] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ade8ce333ea14a3b8533166f1b8c434c, d868bc392e9e47359fbc6137c094dbd7; no voters:
I20260809 12:08:42.675981 29222 raft_consensus.cc:2781] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:08:42.676141 29222 raft_consensus.cc:482] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:08:42.676261 29222 raft_consensus.cc:3037] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:42.678540 29222 raft_consensus.cc:504] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } }
I20260809 12:08:42.679337 29222 leader_election.cc:288] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [CANDIDATE]: Term 1 election: Requested vote from peers 76632b3049d7452eba2a8cf1bd45e545 (127.0.147.67:36945), d868bc392e9e47359fbc6137c094dbd7 (127.0.147.65:46523)
I20260809 12:08:42.679852 29115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7c93060ef24db1ac993f5f1813e859" candidate_uuid: "ade8ce333ea14a3b8533166f1b8c434c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76632b3049d7452eba2a8cf1bd45e545"
I20260809 12:08:42.680088 29115 raft_consensus.cc:3037] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:42.680138 28717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7c93060ef24db1ac993f5f1813e859" candidate_uuid: "ade8ce333ea14a3b8533166f1b8c434c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d868bc392e9e47359fbc6137c094dbd7"
I20260809 12:08:42.680372 28717 raft_consensus.cc:3037] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:42.682416 29115 raft_consensus.cc:2454] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade8ce333ea14a3b8533166f1b8c434c in term 1.
I20260809 12:08:42.682617 28717 raft_consensus.cc:2454] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade8ce333ea14a3b8533166f1b8c434c in term 1.
I20260809 12:08:42.682821 28818 leader_election.cc:302] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76632b3049d7452eba2a8cf1bd45e545, ade8ce333ea14a3b8533166f1b8c434c; no voters:
I20260809 12:08:42.683136 29222 raft_consensus.cc:2781] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:42.683712 29222 raft_consensus.cc:686] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 1 LEADER]: Becoming Leader. State: Replica: ade8ce333ea14a3b8533166f1b8c434c, State: Running, Role: LEADER
I20260809 12:08:42.684228 29222 consensus_queue.cc:237] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } }
I20260809 12:08:42.688757 28510 catalog_manager.cc:5495] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c reported cstate change: term changed from 0 to 1, leader changed from <none> to ade8ce333ea14a3b8533166f1b8c434c (127.0.147.66). New cstate: current_term: 1 leader_uuid: "ade8ce333ea14a3b8533166f1b8c434c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade8ce333ea14a3b8533166f1b8c434c" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260809 12:08:42.764853 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.759658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50968 (local address 127.0.147.66:34911)
0809 12:08:42.759892 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.759897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.759914 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:08:42.760109 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.760113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.760192 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.760371 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:08:42.760678 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.761402 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.762739 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.763164 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.763255 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.763761 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:08:42.764263 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:08:42.764383 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.764470 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.764536 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:08:42.772946 29115 raft_consensus.cc:1261] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [term 1 FOLLOWER]: Refusing update from remote peer ade8ce333ea14a3b8533166f1b8c434c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:42.772943 28717 raft_consensus.cc:1261] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [term 1 FOLLOWER]: Refusing update from remote peer ade8ce333ea14a3b8533166f1b8c434c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:42.773960 29229 consensus_queue.cc:1035] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d868bc392e9e47359fbc6137c094dbd7" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 46523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:42.774360 29222 consensus_queue.cc:1035] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76632b3049d7452eba2a8cf1bd45e545" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 36945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:42.800173 28510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:42.800475 28510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:42.804090 28510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a254217baf1f41dcbad129a8f2e657a3: Sending DeleteTablet for 3 replicas of tablet 8d7c93060ef24db1ac993f5f1813e859
I20260809 12:08:42.805409 28688 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7c93060ef24db1ac993f5f1813e859 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:42 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:39742
I20260809 12:08:42.805423 28882 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7c93060ef24db1ac993f5f1813e859 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:42 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:50954
I20260809 12:08:42.805435 29084 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7c93060ef24db1ac993f5f1813e859 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:42 UTC) from {username='kudu', principal='kudu/127.0.147.126@KRBTEST.COM'} at 127.0.0.1:47158
I20260809 12:08:42.807957 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 28609
I20260809 12:08:42.820161 29246 tablet_replica.cc:330] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: stopping tablet replica
I20260809 12:08:42.825078 29246 raft_consensus.cc:2229] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:42.826146 29246 raft_consensus.cc:2258] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:42.827631 29248 tablet_replica.cc:330] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: stopping tablet replica
I20260809 12:08:42.827823 29247 tablet_replica.cc:330] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: stopping tablet replica
I20260809 12:08:42.828704 29247 raft_consensus.cc:2229] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:08:42.828732 29248 raft_consensus.cc:2229] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:42.829231 29248 raft_consensus.cc:2258] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:42.829380 29247 raft_consensus.cc:2258] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:42.830888 29246 ts_tablet_manager.cc:1905] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:42.831089 29248 ts_tablet_manager.cc:1905] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:42.832036 29247 ts_tablet_manager.cc:1905] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:42.838052 29248 ts_tablet_manager.cc:1918] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:42.838231 29248 log.cc:1196] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/8d7c93060ef24db1ac993f5f1813e859
I20260809 12:08:42.838788 29248 ts_tablet_manager.cc:1939] T 8d7c93060ef24db1ac993f5f1813e859 P 76632b3049d7452eba2a8cf1bd45e545: Deleting consensus metadata
I20260809 12:08:42.839403 29247 ts_tablet_manager.cc:1918] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:42.839651 29247 log.cc:1196] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/8d7c93060ef24db1ac993f5f1813e859
I20260809 12:08:42.840253 29247 ts_tablet_manager.cc:1939] T 8d7c93060ef24db1ac993f5f1813e859 P ade8ce333ea14a3b8533166f1b8c434c: Deleting consensus metadata
I20260809 12:08:42.840948 28493 catalog_manager.cc:4841] TS 76632b3049d7452eba2a8cf1bd45e545 (127.0.147.67:36945): tablet 8d7c93060ef24db1ac993f5f1813e859 (table test-table [id=6b96d0b9249440069c48c5e4f21484bf]) successfully deleted
Aug 09 12:08:42 dist-test-slave-sb2z krb5kdc[28448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.65@KRBTEST.COM
I20260809 12:08:42.841974 28493 catalog_manager.cc:4841] TS ade8ce333ea14a3b8533166f1b8c434c (127.0.147.66:34911): tablet 8d7c93060ef24db1ac993f5f1813e859 (table test-table [id=6b96d0b9249440069c48c5e4f21484bf]) successfully deleted
I20260809 12:08:42.842610 29246 ts_tablet_manager.cc:1918] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:42.842834 29246 log.cc:1196] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786276990256303-589-0/minicluster-data/ts-0/wal/wals/8d7c93060ef24db1ac993f5f1813e859
I20260809 12:08:42.843407 29246 ts_tablet_manager.cc:1939] T 8d7c93060ef24db1ac993f5f1813e859 P d868bc392e9e47359fbc6137c094dbd7: Deleting consensus metadata
I20260809 12:08:42.846313 28488 catalog_manager.cc:4841] TS d868bc392e9e47359fbc6137c094dbd7 (127.0.147.65:46523): tablet 8d7c93060ef24db1ac993f5f1813e859 (table test-table [id=6b96d0b9249440069c48c5e4f21484bf]) successfully deleted
I20260809 12:08:42.849053 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.809584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.0.147.65:46523)
0809 12:08:42.809809 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.809818 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.814418 (+ 4600us) server_negotiation.cc:407] Connection header received
0809 12:08:42.815314 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.815323 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.815440 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.815557 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:42.820534 (+ 4977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.822088 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.826901 (+ 4813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.827608 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.827721 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.842491 (+ 14770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:42.842503 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:42.842516 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:42.842561 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:42.845873 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:42.846802 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:42.846809 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:42.846811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:42.846945 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:42.847361 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:42.847365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:42.847367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:42.847799 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:42.848136 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.848459 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.848684 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:08:42.849730 28774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:42.939813 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 28609
I20260809 12:08:42.966256 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 28789
Aug 09 12:08:42 dist-test-slave-sb2z krb5kdc[28448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.66@KRBTEST.COM
I20260809 12:08:42.987735 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:42.967249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50976 (local address 127.0.147.66:34911)
0809 12:08:42.967468 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:42.967474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:42.967496 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:08:42.967581 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:42.967587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:42.967677 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:42.967789 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:42.969086 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.970328 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.971760 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:42.972243 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:42.972390 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:42.981515 (+ 9125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:42.981531 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:42.981545 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:42.981580 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:42.984791 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:42.985683 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:42.985691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:42.985693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:42.985831 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:42.986246 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:42.986249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:42.986251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:42.986585 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:42.986847 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:42.987177 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 12:08:42.987346 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:08:42.988525 28970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:43.106033 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 28789
I20260809 12:08:43.132642 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 28986
Aug 09 12:08:43 dist-test-slave-sb2z krb5kdc[28448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.67@KRBTEST.COM
I20260809 12:08:43.147951 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:43.133468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47174 (local address 127.0.147.67:36945)
0809 12:08:43.133653 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:43.133658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:43.133677 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:08:43.133768 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:43.133772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:43.133833 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:43.133918 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:43.135325 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:43.136111 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:43.137408 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:43.137803 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:43.137898 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:43.142316 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:43.142330 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:43.142346 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:43.142378 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:43.145052 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:43.145768 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:43.145776 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:43.145778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:43.145919 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:43.146305 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:43.146312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:43.146315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:43.146677 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:43.146993 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:43.147308 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 12:08:43.147665 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:08:43.148625 29171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:43.238350 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 28986
I20260809 12:08:43.265022 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 28467
I20260809 12:08:43.266139 28554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:43.379429 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 28467
2026-08-09T12:08:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5023 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T12:08:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:43Z Disabled control of system clock
I20260809 12:08:43.483136 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:34845
--webserver_interface=127.0.147.126
--webserver_port=0
--builtin_ntp_servers=127.0.147.84:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:34845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:08:43.824841 29260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:43.825114 29260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:43.825193 29260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:43.832159 29260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:43.832321 29260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:43.832402 29260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:43.832463 29260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:43.832507 29260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:43.841313 29260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:34845
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:34845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:43.842557 29260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:43.844332 29260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:43.856014 29270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:43.856048 29271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:43.856103 29274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:43.857043 29260 server_base.cc:1034] running on GCE node
I20260809 12:08:43.857833 29260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:43.859186 29260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:43.860433 29260 hybrid_clock.cc:648] HybridClock initialized: now 1786277323860394 us; error 54 us; skew 500 ppm
I20260809 12:08:43.860792 29260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:43.863057 29260 webserver.cc:460] Webserver started at http://127.0.147.126:43913/ using document root <none> and password file <none>
I20260809 12:08:43.863798 29260 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:43.863929 29260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:43.864241 29260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:43.867035 29260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "1e91b95aabbd4c76832d92f69496b3a6"
format_stamp: "Formatted at 2026-08-09 12:08:43 on dist-test-slave-sb2z"
server_key: "7b81084eaa73dda7e310e46107184b9a"
server_key_iv: "6e5a5d08d46675977261b6f17c5573fa"
server_key_version: "encryptionkey@0"
I20260809 12:08:43.867926 29260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "1e91b95aabbd4c76832d92f69496b3a6"
format_stamp: "Formatted at 2026-08-09 12:08:43 on dist-test-slave-sb2z"
server_key: "7b81084eaa73dda7e310e46107184b9a"
server_key_iv: "6e5a5d08d46675977261b6f17c5573fa"
server_key_version: "encryptionkey@0"
I20260809 12:08:43.872951 29260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:08:43.876296 29281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:43.877612 29260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:08:43.877794 29260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "1e91b95aabbd4c76832d92f69496b3a6"
format_stamp: "Formatted at 2026-08-09 12:08:43 on dist-test-slave-sb2z"
server_key: "7b81084eaa73dda7e310e46107184b9a"
server_key_iv: "6e5a5d08d46675977261b6f17c5573fa"
server_key_version: "encryptionkey@0"
I20260809 12:08:43.878002 29260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:43.898303 29260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:43.899120 29260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:43.899416 29260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:43.918233 29260 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:34845
I20260809 12:08:43.918252 29357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:34845 every 8 connection(s)
I20260809 12:08:43.919548 29260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:43.923352 29358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:43.923902 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 29260
I20260809 12:08:43.924093 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:43.924584 589 external_mini_cluster.cc:1428] Setting key 51ab22648059f78dc93ace4b2d3261b0
I20260809 12:08:43.938020 29358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6: Bootstrap starting.
I20260809 12:08:43.940184 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:43.926494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52260 (local address 127.0.147.126:34845)
0809 12:08:43.927584 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:43.927599 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:43.927649 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:08:43.930151 (+ 2502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:43.930181 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:43.930759 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:43.931208 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:43.931755 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:43.933645 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:43.935043 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:43.936132 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:43.936300 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:43.937416 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:43.937429 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:43.937451 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:43.938156 (+ 705us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:43.938253 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:43.938280 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:43.938391 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:43.938593 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 12:08:43.938902 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":326,"threads_started":1}
I20260809 12:08:43.941718 29358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:43.942811 29358 log.cc:824] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:43.945353 29358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6: No bootstrap required, opened a new log
I20260809 12:08:43.952334 29358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e91b95aabbd4c76832d92f69496b3a6" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34845 } }
I20260809 12:08:43.952708 29358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:43.952826 29358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e91b95aabbd4c76832d92f69496b3a6, State: Initialized, Role: FOLLOWER
I20260809 12:08:43.953450 29358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e91b95aabbd4c76832d92f69496b3a6" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34845 } }
I20260809 12:08:43.953703 29358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:43.953856 29358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:43.954020 29358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:43.956236 29358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e91b95aabbd4c76832d92f69496b3a6" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34845 } }
I20260809 12:08:43.956705 29358 leader_election.cc:302] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e91b95aabbd4c76832d92f69496b3a6; no voters:
I20260809 12:08:43.957280 29358 leader_election.cc:288] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:43.957437 29366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:43.958249 29366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1e91b95aabbd4c76832d92f69496b3a6, State: Running, Role: LEADER
I20260809 12:08:43.958868 29366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e91b95aabbd4c76832d92f69496b3a6" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34845 } }
I20260809 12:08:43.959645 29358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:43.962662 29367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e91b95aabbd4c76832d92f69496b3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e91b95aabbd4c76832d92f69496b3a6" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34845 } } }
I20260809 12:08:43.962539 29370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e91b95aabbd4c76832d92f69496b3a6. Latest consensus state: current_term: 1 leader_uuid: "1e91b95aabbd4c76832d92f69496b3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e91b95aabbd4c76832d92f69496b3a6" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34845 } } }
I20260809 12:08:43.963018 29370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:43.963030 29367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:43.964495 29377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:43.969447 29377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:43.978873 29377 catalog_manager.cc:1347] Generated new cluster ID: 8b816edd203649e990eb917788bb3315
I20260809 12:08:43.979040 29377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:44.005008 29377 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:44.006708 29377 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:44.014719 29377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6: Generated new TSK 0
I20260809 12:08:44.015636 29377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:08:44.019960 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.65:0
--local_ip_for_outbound_sockets=127.0.147.65
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34845
--builtin_ntp_servers=127.0.147.84:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:08:44.368497 29393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:44.368846 29393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:44.368996 29393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:44.376602 29393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:44.376691 29393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:44.376853 29393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.65
I20260809 12:08:44.386039 29393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.147.65
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.65
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:44.387256 29393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:44.389038 29393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:44.402215 29401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:44.402215 29405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:44.402251 29402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:44.403437 29393 server_base.cc:1034] running on GCE node
I20260809 12:08:44.404170 29393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:44.405057 29393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:44.406260 29393 hybrid_clock.cc:648] HybridClock initialized: now 1786277324406221 us; error 37 us; skew 500 ppm
I20260809 12:08:44.406541 29393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:44.408705 29393 webserver.cc:460] Webserver started at http://127.0.147.65:42537/ using document root <none> and password file <none>
I20260809 12:08:44.409332 29393 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:44.409421 29393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:44.409694 29393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:44.412462 29393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/data/instance:
uuid: "39c1d66287e941aa8caba55e92ea6026"
format_stamp: "Formatted at 2026-08-09 12:08:44 on dist-test-slave-sb2z"
server_key: "798a0f9e3da01c1533ff67a46cd5301f"
server_key_iv: "d96701711ecfd7394c6fd2b88e5c558f"
server_key_version: "encryptionkey@0"
I20260809 12:08:44.413215 29393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/wal/instance:
uuid: "39c1d66287e941aa8caba55e92ea6026"
format_stamp: "Formatted at 2026-08-09 12:08:44 on dist-test-slave-sb2z"
server_key: "798a0f9e3da01c1533ff67a46cd5301f"
server_key_iv: "d96701711ecfd7394c6fd2b88e5c558f"
server_key_version: "encryptionkey@0"
I20260809 12:08:44.418149 29393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:08:44.421375 29412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:44.422618 29393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 12:08:44.422796 29393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/wal
uuid: "39c1d66287e941aa8caba55e92ea6026"
format_stamp: "Formatted at 2026-08-09 12:08:44 on dist-test-slave-sb2z"
server_key: "798a0f9e3da01c1533ff67a46cd5301f"
server_key_iv: "d96701711ecfd7394c6fd2b88e5c558f"
server_key_version: "encryptionkey@0"
I20260809 12:08:44.423010 29393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:44.443346 29393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:44.444584 29393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:44.445012 29393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:44.446028 29393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:44.447599 29393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:44.447728 29393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:44.447845 29393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:44.447947 29393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:44.482738 29393 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.65:40745
I20260809 12:08:44.482849 29576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.65:40745 every 8 connection(s)
I20260809 12:08:44.484109 29393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/data/info.pb
I20260809 12:08:44.485635 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 29393
I20260809 12:08:44.485814 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-0/wal/instance
I20260809 12:08:44.486377 589 external_mini_cluster.cc:1428] Setting key 53a025b4178a363f19d54d8e46ff1a35
I20260809 12:08:44.489753 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.66:0
--local_ip_for_outbound_sockets=127.0.147.66
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34845
--builtin_ntp_servers=127.0.147.84:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:08:44.499296 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:44.486897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.65:48987 (local address 127.0.147.126:34845)
0809 12:08:44.487407 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:44.487413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:44.488885 (+ 1472us) server_negotiation.cc:407] Connection header received
0809 12:08:44.491713 (+ 2828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:44.491717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:44.491782 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:44.491876 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:44.493368 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:44.494096 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:44.496502 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:44.496886 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:44.496980 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:44.498581 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:44.498586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:44.498589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:44.498607 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:44.498646 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:44.498658 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:44.498727 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:44.498982 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 12:08:44.499055 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":266,"threads_started":1}
I20260809 12:08:44.500773 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:44.487617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:34845 (local address 127.0.147.65:48987)
0809 12:08:44.488706 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:44.488763 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:44.491386 (+ 2623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:44.492140 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:44.492155 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:44.492270 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:44.493086 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:44.493113 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:44.494255 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:44.494263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:44.496364 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:44.496382 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:44.497107 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:44.497116 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:44.497449 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:44.498225 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:08:44.498231 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:44.498280 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:08:44.498385 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:44.498758 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:44.498773 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:44.498784 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 12:08:44.498971 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 12:08:44.499382 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":315,"threads_started":1}
I20260809 12:08:44.502310 29577 heartbeater.cc:347] Connected to a master server at 127.0.147.126:34845
I20260809 12:08:44.502636 29577 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:44.503552 29577 heartbeater.cc:511] Master 127.0.147.126:34845 requested a full tablet report, sending...
I20260809 12:08:44.505899 29304 ts_manager.cc:194] Registered new tserver with Master: 39c1d66287e941aa8caba55e92ea6026 (127.0.147.65:40745)
I20260809 12:08:44.507431 29304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.65:48987
W20260809 12:08:44.837949 29583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:44.838268 29583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:44.838434 29583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:44.845746 29583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:44.845871 29583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:44.846045 29583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.66
I20260809 12:08:44.854861 29583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.147.66
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.66
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:44.856323 29583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:44.858068 29583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:44.871294 29591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:44.871344 29595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:44.871269 29593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:44.872097 29583 server_base.cc:1034] running on GCE node
I20260809 12:08:44.872689 29583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:44.873615 29583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:44.874835 29583 hybrid_clock.cc:648] HybridClock initialized: now 1786277324874791 us; error 46 us; skew 500 ppm
I20260809 12:08:44.875162 29583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:44.877554 29583 webserver.cc:460] Webserver started at http://127.0.147.66:34115/ using document root <none> and password file <none>
I20260809 12:08:44.878235 29583 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:44.878356 29583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:44.878703 29583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:44.881762 29583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/data/instance:
uuid: "4879918df566490baacefc5570451cac"
format_stamp: "Formatted at 2026-08-09 12:08:44 on dist-test-slave-sb2z"
server_key: "7c49aa868949595641eeb67eb5981b70"
server_key_iv: "6607f0b9528512fdc3582903d8927356"
server_key_version: "encryptionkey@0"
I20260809 12:08:44.882612 29583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/wal/instance:
uuid: "4879918df566490baacefc5570451cac"
format_stamp: "Formatted at 2026-08-09 12:08:44 on dist-test-slave-sb2z"
server_key: "7c49aa868949595641eeb67eb5981b70"
server_key_iv: "6607f0b9528512fdc3582903d8927356"
server_key_version: "encryptionkey@0"
I20260809 12:08:44.887727 29583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 12:08:44.891085 29607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:44.892436 29583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:08:44.892621 29583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/wal
uuid: "4879918df566490baacefc5570451cac"
format_stamp: "Formatted at 2026-08-09 12:08:44 on dist-test-slave-sb2z"
server_key: "7c49aa868949595641eeb67eb5981b70"
server_key_iv: "6607f0b9528512fdc3582903d8927356"
server_key_version: "encryptionkey@0"
I20260809 12:08:44.892839 29583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:44.917057 29583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:44.918169 29583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:44.918583 29583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:44.919835 29583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:44.921747 29583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:44.921907 29583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:44.922013 29583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:44.922089 29583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:44.963600 29583 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.66:37847
I20260809 12:08:44.963709 29765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.66:37847 every 8 connection(s)
I20260809 12:08:44.964983 29583 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/data/info.pb
I20260809 12:08:44.967906 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 29583
I20260809 12:08:44.968122 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/wal/instance
I20260809 12:08:44.968963 589 external_mini_cluster.cc:1428] Setting key 566380aca363737c6bc49c549fb2315a
I20260809 12:08:44.972190 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.147.67:0
--local_ip_for_outbound_sockets=127.0.147.67
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34845
--builtin_ntp_servers=127.0.147.84:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:08:44.979015 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:44.967614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.66:39571 (local address 127.0.147.126:34845)
0809 12:08:44.967796 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:44.967800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:44.969748 (+ 1948us) server_negotiation.cc:407] Connection header received
0809 12:08:44.971887 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:44.971895 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:44.972001 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:44.972119 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:44.973202 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:44.974396 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:44.975968 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:44.976686 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:44.976836 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:44.978116 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:44.978124 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:44.978132 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:44.978157 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:44.978201 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:44.978219 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:44.978314 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:44.978511 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 12:08:44.978622 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 12:08:44.979854 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:44.968302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:34845 (local address 127.0.147.66:39571)
0809 12:08:44.969586 (+ 1284us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:44.969631 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:44.971602 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:44.972336 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:44.972351 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:44.972473 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:44.973023 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:44.973039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:44.974544 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:44.974547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:44.975819 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:44.975832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:44.976892 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:44.976897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:44.977098 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:44.977802 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:08:44.977805 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:44.977848 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:08:44.977924 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:44.978317 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:44.978325 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:44.978330 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:08:44.978466 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 12:08:44.978743 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":458,"threads_started":1}
I20260809 12:08:44.981320 29766 heartbeater.cc:347] Connected to a master server at 127.0.147.126:34845
I20260809 12:08:44.981623 29766 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:44.982383 29766 heartbeater.cc:511] Master 127.0.147.126:34845 requested a full tablet report, sending...
I20260809 12:08:44.983801 29304 ts_manager.cc:194] Registered new tserver with Master: 4879918df566490baacefc5570451cac (127.0.147.66:37847)
I20260809 12:08:44.984580 29304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.66:39571
W20260809 12:08:45.328274 29771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:45.328562 29771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:45.328653 29771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:45.335807 29771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:45.335898 29771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:45.336001 29771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.147.67
I20260809 12:08:45.344959 29771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.147.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.147.67
--webserver_port=0
--tserver_master_addrs=127.0.147.126:34845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.147.67
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:45.346216 29771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:45.347939 29771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:45.361557 29778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:45.361588 29781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:45.361661 29776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:45.362167 29771 server_base.cc:1034] running on GCE node
I20260809 12:08:45.362886 29771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:45.363824 29771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:45.365069 29771 hybrid_clock.cc:648] HybridClock initialized: now 1786277325364994 us; error 65 us; skew 500 ppm
I20260809 12:08:45.365430 29771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:08:45.367882 29771 webserver.cc:460] Webserver started at http://127.0.147.67:46143/ using document root <none> and password file <none>
I20260809 12:08:45.368628 29771 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:45.368748 29771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:45.369091 29771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:45.372095 29771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/data/instance:
uuid: "e068527ff4d74d97a67ad493d584e96e"
format_stamp: "Formatted at 2026-08-09 12:08:45 on dist-test-slave-sb2z"
server_key: "36b9ac163ebb03bdd14f475818bec729"
server_key_iv: "68d90411758e3c1e0fc323df1501084c"
server_key_version: "encryptionkey@0"
I20260809 12:08:45.372853 29771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/wal/instance:
uuid: "e068527ff4d74d97a67ad493d584e96e"
format_stamp: "Formatted at 2026-08-09 12:08:45 on dist-test-slave-sb2z"
server_key: "36b9ac163ebb03bdd14f475818bec729"
server_key_iv: "68d90411758e3c1e0fc323df1501084c"
server_key_version: "encryptionkey@0"
I20260809 12:08:45.377959 29771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260809 12:08:45.381445 29791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:45.382817 29771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:08:45.382951 29771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/wal
uuid: "e068527ff4d74d97a67ad493d584e96e"
format_stamp: "Formatted at 2026-08-09 12:08:45 on dist-test-slave-sb2z"
server_key: "36b9ac163ebb03bdd14f475818bec729"
server_key_iv: "68d90411758e3c1e0fc323df1501084c"
server_key_version: "encryptionkey@0"
I20260809 12:08:45.383132 29771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:45.417344 29771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:45.418126 29771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:45.418382 29771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:45.419324 29771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:08:45.420753 29771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:08:45.420843 29771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:45.420938 29771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:08:45.421000 29771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:45.456017 29771 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.67:39933
I20260809 12:08:45.456110 29969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.67:39933 every 8 connection(s)
I20260809 12:08:45.457304 29771 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/data/info.pb
I20260809 12:08:45.458850 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 29771
I20260809 12:08:45.459045 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/wal/instance
I20260809 12:08:45.459703 589 external_mini_cluster.cc:1428] Setting key 1c93863c14912997fb656d723294ed03
I20260809 12:08:45.471218 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.461526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:40215 (local address 127.0.147.126:34845)
0809 12:08:45.461713 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.461719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.462105 (+ 386us) server_negotiation.cc:407] Connection header received
0809 12:08:45.464029 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.464035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.464141 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.464259 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:45.465467 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.466534 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.468271 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.468798 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.468946 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.470382 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:45.470388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:45.470393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:45.470418 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:45.470451 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:45.470462 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:45.470560 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.470785 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.470882 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:08:45.472273 29974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.460829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:34845 (local address 127.0.147.67:40215)
0809 12:08:45.461932 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.461976 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:45.463754 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:45.464498 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:45.464506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:45.464601 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:08:45.465272 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.465300 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.466709 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.466714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.468119 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.468132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.469048 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.469056 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.469288 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.470063 (+ 775us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:08:45.470067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:08:45.470104 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:08:45.470184 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:08:45.470567 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:08:45.470581 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:08:45.470591 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:08:45.470772 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 12:08:45.471144 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":385,"threads_started":1}
I20260809 12:08:45.473649 29971 heartbeater.cc:347] Connected to a master server at 127.0.147.126:34845
I20260809 12:08:45.473899 29971 heartbeater.cc:464] Registering TS with master...
I20260809 12:08:45.474570 29971 heartbeater.cc:511] Master 127.0.147.126:34845 requested a full tablet report, sending...
I20260809 12:08:45.475953 29304 ts_manager.cc:194] Registered new tserver with Master: e068527ff4d74d97a67ad493d584e96e (127.0.147.67:39933)
I20260809 12:08:45.476768 29304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.147.67:40215
I20260809 12:08:45.479943 589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:08:45.491179 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.484494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.0.147.126:34845)
0809 12:08:45.484699 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.484705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.485273 (+ 568us) server_negotiation.cc:407] Connection header received
0809 12:08:45.485435 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.485439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.485508 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.485595 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:45.486214 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.487091 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.489030 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.489443 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.489536 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.490505 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:45.490510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:45.490513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:45.490529 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:45.490559 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:45.490573 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:45.490641 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.490764 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.490839 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:08:45.494825 29304 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:08:45.501757 29304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:08:45.511067 29577 heartbeater.cc:503] Master 127.0.147.126:34845 was elected leader, sending a full tablet report...
I20260809 12:08:45.525374 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.514156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:37847 (local address 127.0.0.1:41382)
0809 12:08:45.514998 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.515073 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:45.515286 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:45.517309 (+ 2023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:45.517318 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:45.517360 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:45.517755 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.517770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.520079 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.520084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.522663 (+ 2579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.522686 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.524576 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.524582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.524734 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.524760 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 12:08:45.524838 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 12:08:45.524934 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":1,"thread_start_us":314,"threads_started":1}
I20260809 12:08:45.525815 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.515771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36862 (local address 127.0.147.67:39933)
0809 12:08:45.517632 (+ 1861us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.517643 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.517681 (+ 38us) server_negotiation.cc:407] Connection header received
0809 12:08:45.517785 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.517794 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.518017 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.518859 (+ 842us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:45.519138 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.520740 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.522801 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.524044 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.524874 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.525192 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.525269 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.525444 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1668,"thread_start_us":2103,"threads_started":1}
I20260809 12:08:45.525399 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.515243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.67:39933 (local address 127.0.0.1:36862)
0809 12:08:45.516935 (+ 1692us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.516953 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:45.517062 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:45.518237 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:45.518242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:45.518270 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:45.518650 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.518663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.520946 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.520951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.522603 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.522621 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.524266 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.524272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.524505 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.524579 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 12:08:45.524689 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 12:08:45.524878 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1583,"mutex_wait_us":3,"thread_start_us":1457,"threads_started":1}
I20260809 12:08:45.526364 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.515394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.0.147.65:40745)
0809 12:08:45.518523 (+ 3129us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.518530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.518551 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:08:45.518625 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.518630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.518759 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.518969 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:45.519471 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.521031 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.523653 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.525097 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.525401 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.525685 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.525817 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.526016 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2987,"thread_start_us":2854,"threads_started":1}
I20260809 12:08:45.526475 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.514845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.0.1:60000)
0809 12:08:45.515956 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.515979 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:45.516117 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:45.518942 (+ 2825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:45.518947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:45.518972 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:45.519317 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.519328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.521205 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.521210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.523405 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.523419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.525435 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.525445 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.525646 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.525674 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 12:08:45.525764 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 12:08:45.526151 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"thread_start_us":829,"threads_started":1}
I20260809 12:08:45.526928 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.514421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41382 (local address 127.0.147.66:37847)
0809 12:08:45.516605 (+ 2184us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.516616 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.516653 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:08:45.516761 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.516770 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.517060 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.517946 (+ 886us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:45.518237 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.519909 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.522915 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.524371 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.524508 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.524754 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.526380 (+ 1626us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.526565 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2038,"thread_start_us":1930,"threads_started":1}
I20260809 12:08:45.530089 29867 tablet_service.cc:1467] Processing CreateTablet for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9 (DEFAULT_TABLE table=test-table [id=1ca30dcdba9e4f62993fea68a04f8f18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:45.530747 29675 tablet_service.cc:1467] Processing CreateTablet for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9 (DEFAULT_TABLE table=test-table [id=1ca30dcdba9e4f62993fea68a04f8f18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:45.530752 29483 tablet_service.cc:1467] Processing CreateTablet for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9 (DEFAULT_TABLE table=test-table [id=1ca30dcdba9e4f62993fea68a04f8f18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:08:45.531481 29867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:45.532135 29483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:45.532124 29675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:45.543273 30004 tablet_bootstrap.cc:492] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: Bootstrap starting.
I20260809 12:08:45.547235 30006 tablet_bootstrap.cc:492] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026: Bootstrap starting.
I20260809 12:08:45.548023 30004 tablet_bootstrap.cc:654] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:45.549055 30005 tablet_bootstrap.cc:492] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: Bootstrap starting.
I20260809 12:08:45.549793 30004 log.cc:824] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:45.551052 30006 tablet_bootstrap.cc:654] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:45.552378 30005 tablet_bootstrap.cc:654] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:45.552392 30006 log.cc:824] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:45.553359 30004 tablet_bootstrap.cc:492] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: No bootstrap required, opened a new log
I20260809 12:08:45.553562 30005 log.cc:824] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:45.553648 30004 ts_tablet_manager.cc:1397] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260809 12:08:45.556048 30006 tablet_bootstrap.cc:492] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026: No bootstrap required, opened a new log
I20260809 12:08:45.556394 30006 ts_tablet_manager.cc:1397] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260809 12:08:45.557825 30005 tablet_bootstrap.cc:492] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: No bootstrap required, opened a new log
I20260809 12:08:45.558133 30005 ts_tablet_manager.cc:1397] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 12:08:45.562451 30004 raft_consensus.cc:348] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } }
I20260809 12:08:45.562840 30004 raft_consensus.cc:374] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:45.562947 30004 raft_consensus.cc:729] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e068527ff4d74d97a67ad493d584e96e, State: Initialized, Role: FOLLOWER
I20260809 12:08:45.563616 30004 consensus_queue.cc:260] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } }
I20260809 12:08:45.564726 29971 heartbeater.cc:503] Master 127.0.147.126:34845 was elected leader, sending a full tablet report...
I20260809 12:08:45.565444 30004 ts_tablet_manager.cc:1428] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: Time spent starting tablet: real 0.012s user 0.005s sys 0.004s
I20260809 12:08:45.568460 30006 raft_consensus.cc:348] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } }
I20260809 12:08:45.569016 30006 raft_consensus.cc:374] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:45.569140 30006 raft_consensus.cc:729] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c1d66287e941aa8caba55e92ea6026, State: Initialized, Role: FOLLOWER
I20260809 12:08:45.569808 30006 consensus_queue.cc:260] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } }
I20260809 12:08:45.570155 30005 raft_consensus.cc:348] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } }
I20260809 12:08:45.570725 30005 raft_consensus.cc:374] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:45.570844 30005 raft_consensus.cc:729] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4879918df566490baacefc5570451cac, State: Initialized, Role: FOLLOWER
I20260809 12:08:45.571898 30006 ts_tablet_manager.cc:1428] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026: Time spent starting tablet: real 0.015s user 0.014s sys 0.002s
I20260809 12:08:45.571633 30005 consensus_queue.cc:260] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } }
I20260809 12:08:45.572968 29766 heartbeater.cc:503] Master 127.0.147.126:34845 was elected leader, sending a full tablet report...
I20260809 12:08:45.573794 30005 ts_tablet_manager.cc:1428] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260809 12:08:45.606101 30012 raft_consensus.cc:482] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:08:45.606433 30012 raft_consensus.cc:504] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } }
I20260809 12:08:45.608410 30012 leader_election.cc:288] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4879918df566490baacefc5570451cac (127.0.147.66:37847), 39c1d66287e941aa8caba55e92ea6026 (127.0.147.65:40745)
I20260809 12:08:45.614781 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.608741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:48773 (local address 127.0.147.66:37847)
0809 12:08:45.608969 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.608975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.609095 (+ 120us) server_negotiation.cc:407] Connection header received
0809 12:08:45.609331 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.609340 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.609451 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.609551 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:45.610053 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.611019 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.612982 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.613939 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.614069 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.614119 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.614435 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.614520 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:08:45.614877 29974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.608770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.66:37847 (local address 127.0.147.67:48773)
0809 12:08:45.608982 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.609004 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:45.609146 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:45.609634 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:45.609640 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:45.609672 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:45.609889 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.609903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.611171 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.611177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.612824 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.612836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.614085 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.614091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.614263 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.614301 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:08:45.614394 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 12:08:45.614548 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":3}
I20260809 12:08:45.615459 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.609120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.147.67:53741 (local address 127.0.147.65:40745)
0809 12:08:45.609375 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.609385 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.609859 (+ 474us) server_negotiation.cc:407] Connection header received
0809 12:08:45.609945 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.609953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.610065 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.610165 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:08:45.610749 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.611670 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.613595 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.614588 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.614750 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.614810 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.615147 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.615279 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:08:45.616057 30015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.608935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.147.67:53741)
0809 12:08:45.609747 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.609761 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:08:45.609859 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:08:45.610195 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:08:45.610199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:08:45.610226 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:08:45.610580 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.610592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.611814 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.611817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.613395 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:08:45.613409 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.614759 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.614767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:08:45.614965 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.614994 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 12:08:45.615675 (+ 681us) client_negotiation.cc:240] Negotiation successful
0809 12:08:45.615816 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":467,"threads_started":1}
I20260809 12:08:45.616183 29704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea7cbd2ffe994f518eb5bb9fa5eda9f9" candidate_uuid: "e068527ff4d74d97a67ad493d584e96e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4879918df566490baacefc5570451cac" is_pre_election: true
I20260809 12:08:45.616830 29704 raft_consensus.cc:2454] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e068527ff4d74d97a67ad493d584e96e in term 0.
I20260809 12:08:45.617245 29513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea7cbd2ffe994f518eb5bb9fa5eda9f9" candidate_uuid: "e068527ff4d74d97a67ad493d584e96e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c1d66287e941aa8caba55e92ea6026" is_pre_election: true
I20260809 12:08:45.617520 29796 leader_election.cc:302] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4879918df566490baacefc5570451cac, e068527ff4d74d97a67ad493d584e96e; no voters:
I20260809 12:08:45.617807 29513 raft_consensus.cc:2454] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e068527ff4d74d97a67ad493d584e96e in term 0.
I20260809 12:08:45.617954 30012 raft_consensus.cc:2781] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:08:45.618112 30012 raft_consensus.cc:482] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:08:45.618225 30012 raft_consensus.cc:3037] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:45.620615 30012 raft_consensus.cc:504] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } }
I20260809 12:08:45.621294 30012 leader_election.cc:288] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [CANDIDATE]: Term 1 election: Requested vote from peers 4879918df566490baacefc5570451cac (127.0.147.66:37847), 39c1d66287e941aa8caba55e92ea6026 (127.0.147.65:40745)
I20260809 12:08:45.621701 29704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea7cbd2ffe994f518eb5bb9fa5eda9f9" candidate_uuid: "e068527ff4d74d97a67ad493d584e96e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4879918df566490baacefc5570451cac"
I20260809 12:08:45.621745 29513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea7cbd2ffe994f518eb5bb9fa5eda9f9" candidate_uuid: "e068527ff4d74d97a67ad493d584e96e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c1d66287e941aa8caba55e92ea6026"
I20260809 12:08:45.621924 29704 raft_consensus.cc:3037] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:45.621923 29513 raft_consensus.cc:3037] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:45.625254 29704 raft_consensus.cc:2454] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e068527ff4d74d97a67ad493d584e96e in term 1.
I20260809 12:08:45.625243 29513 raft_consensus.cc:2454] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e068527ff4d74d97a67ad493d584e96e in term 1.
I20260809 12:08:45.625669 29792 leader_election.cc:302] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c1d66287e941aa8caba55e92ea6026, e068527ff4d74d97a67ad493d584e96e; no voters:
I20260809 12:08:45.626034 30012 raft_consensus.cc:2781] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:45.626487 30012 raft_consensus.cc:686] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 1 LEADER]: Becoming Leader. State: Replica: e068527ff4d74d97a67ad493d584e96e, State: Running, Role: LEADER
I20260809 12:08:45.627034 30012 consensus_queue.cc:237] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } }
I20260809 12:08:45.631692 29304 catalog_manager.cc:5495] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e reported cstate change: term changed from 0 to 1, leader changed from <none> to e068527ff4d74d97a67ad493d584e96e (127.0.147.67). New cstate: current_term: 1 leader_uuid: "e068527ff4d74d97a67ad493d584e96e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e068527ff4d74d97a67ad493d584e96e" member_type: VOTER last_known_addr { host: "127.0.147.67" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 } health_report { overall_health: UNKNOWN } } }
I20260809 12:08:45.651279 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.646271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.0.147.67:39933)
0809 12:08:45.646499 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.646504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.646527 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:08:45.646680 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.646683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.646773 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.646895 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:08:45.647225 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.648028 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.649330 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.649720 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.649809 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.650232 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:08:45.650714 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:08:45.650820 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.650928 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.650986 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 12:08:45.659373 29513 raft_consensus.cc:1261] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [term 1 FOLLOWER]: Refusing update from remote peer e068527ff4d74d97a67ad493d584e96e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:45.659374 29704 raft_consensus.cc:1261] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [term 1 FOLLOWER]: Refusing update from remote peer e068527ff4d74d97a67ad493d584e96e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:08:45.660456 30018 consensus_queue.cc:1035] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4879918df566490baacefc5570451cac" member_type: VOTER last_known_addr { host: "127.0.147.66" port: 37847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:45.660822 30012 consensus_queue.cc:1035] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39c1d66287e941aa8caba55e92ea6026" member_type: VOTER last_known_addr { host: "127.0.147.65" port: 40745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:08:45.674460 30023 mvcc.cc:205] Tried to move back new op lower bound from 7316591925890744320 to 7316591925773099008. Current Snapshot: MvccSnapshot[applied={T|T < 7316591925890744320}]
I20260809 12:08:45.673717 30022 mvcc.cc:205] Tried to move back new op lower bound from 7316591925890744320 to 7316591925773099008. Current Snapshot: MvccSnapshot[applied={T|T < 7316591925890744320}]
I20260809 12:08:45.690301 29304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:45.690546 29304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:08:45.694197 29304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e91b95aabbd4c76832d92f69496b3a6: Sending DeleteTablet for 3 replicas of tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9
I20260809 12:08:45.695443 29675 tablet_service.cc:1514] Processing DeleteTablet for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:45 UTC) from {username='slave'} at 127.0.0.1:41382
I20260809 12:08:45.695443 29483 tablet_service.cc:1514] Processing DeleteTablet for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:45 UTC) from {username='slave'} at 127.0.0.1:60000
I20260809 12:08:45.695883 29867 tablet_service.cc:1514] Processing DeleteTablet for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:08:45 UTC) from {username='slave'} at 127.0.0.1:36862
I20260809 12:08:45.697253 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 29393
I20260809 12:08:45.723991 30037 tablet_replica.cc:330] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: stopping tablet replica
I20260809 12:08:45.726823 30037 raft_consensus.cc:2229] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:08:45.729012 30037 raft_consensus.cc:2258] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:45.733455 30037 ts_tablet_manager.cc:1905] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:45.734670 30036 tablet_replica.cc:330] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: stopping tablet replica
I20260809 12:08:45.736824 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.698598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.0.147.65:40745)
0809 12:08:45.698780 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.698786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.722074 (+ 23288us) server_negotiation.cc:407] Connection header received
0809 12:08:45.722568 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.722575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.722713 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.722793 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:45.724239 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.725433 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.729448 (+ 4015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.730168 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.730270 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.734968 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:45.734981 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:45.735000 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:45.735838 (+ 838us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:45.735904 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:45.735931 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:45.736034 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.736300 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.736422 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:08:45.738199 29570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:45.738508 30036 raft_consensus.cc:2229] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:45.739229 30036 raft_consensus.cc:2258] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:45.741715 30036 ts_tablet_manager.cc:1905] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:08:45.744403 30037 ts_tablet_manager.cc:1918] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:45.744691 30037 log.cc:1196] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-2/wal/wals/ea7cbd2ffe994f518eb5bb9fa5eda9f9
I20260809 12:08:45.745453 30037 ts_tablet_manager.cc:1939] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P e068527ff4d74d97a67ad493d584e96e: Deleting consensus metadata
I20260809 12:08:45.747855 29287 catalog_manager.cc:4841] TS e068527ff4d74d97a67ad493d584e96e (127.0.147.67:39933): tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9 (table test-table [id=1ca30dcdba9e4f62993fea68a04f8f18]) successfully deleted
I20260809 12:08:45.750990 30036 ts_tablet_manager.cc:1918] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:08:45.751207 30036 log.cc:1196] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: Deleting WAL directory at /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786276990256303-589-0/minicluster-data/ts-1/wal/wals/ea7cbd2ffe994f518eb5bb9fa5eda9f9
I20260809 12:08:45.751971 30036 ts_tablet_manager.cc:1939] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 4879918df566490baacefc5570451cac: Deleting consensus metadata
I20260809 12:08:45.754096 29287 catalog_manager.cc:4841] TS 4879918df566490baacefc5570451cac (127.0.147.66:37847): tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9 (table test-table [id=1ca30dcdba9e4f62993fea68a04f8f18]) successfully deleted
I20260809 12:08:45.846249 30035 tablet_replica.cc:330] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026: stopping tablet replica
I20260809 12:08:45.846776 30035 raft_consensus.cc:2229] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:08:45.847349 30035 raft_consensus.cc:2258] T ea7cbd2ffe994f518eb5bb9fa5eda9f9 P 39c1d66287e941aa8caba55e92ea6026 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:08:45.848415 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 29393
I20260809 12:08:45.874094 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 29583
W20260809 12:08:45.874002 29282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.147.65:40745 (error 108)
I20260809 12:08:45.877457 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.876929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.0.1:60008)
0809 12:08:45.877152 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.877283 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 12:08:45.877743 29282 catalog_manager.cc:4584] TS 39c1d66287e941aa8caba55e92ea6026 (127.0.147.65:40745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea7cbd2ffe994f518eb5bb9fa5eda9f9: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40745: connect: Connection refused (error 111)
I20260809 12:08:45.882331 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.875227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41384 (local address 127.0.147.66:37847)
0809 12:08:45.875430 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:45.875437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:45.875458 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:08:45.875529 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:45.875534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:45.875676 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:45.875795 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:45.876404 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.877359 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.878780 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:45.879443 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:45.879721 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:45.880776 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:45.880786 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:45.880818 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:45.881476 (+ 658us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:45.881548 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:45.881577 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:45.881708 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:45.881850 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 12:08:45.881968 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:08:45.883050 29761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:45.931730 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.931136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.0.1:60022)
0809 12:08:45.931326 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.931444 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 12:08:45.933032 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.932681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.0.1:60036)
0809 12:08:45.932811 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.932886 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 12:08:45.992079 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 29583
I20260809 12:08:45.997936 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.997383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.0.1:60042)
0809 12:08:45.997603 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.997712 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 12:08:45.999850 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:45.999290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.0.1:60058)
0809 12:08:45.999549 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:45.999679 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 12:08:46.018296 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 29771
I20260809 12:08:46.026163 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:46.019183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.0.147.67:39933)
0809 12:08:46.019740 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:46.019745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:46.019760 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:08:46.019815 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:46.019819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:46.019938 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:46.020066 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:46.020528 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:46.021363 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:46.022683 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:46.023143 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:46.023291 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:46.024600 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:46.024610 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:46.024626 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:46.025277 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:46.025326 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:46.025346 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:46.025495 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:46.025639 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 12:08:46.025743 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437}
I20260809 12:08:46.027148 29963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:46.080502 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:46.080036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.0.1:60064)
0809 12:08:46.080249 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:46.080339 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 12:08:46.081633 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:46.081196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.0.1:60074)
0809 12:08:46.081448 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:46.081519 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 12:08:46.155161 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 29771
I20260809 12:08:46.182338 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 29260
I20260809 12:08:46.183602 29350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:46.293931 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 29260
I20260809 12:08:46.293979 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:46.293466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.65:40745 (local address 127.0.0.1:60090)
0809 12:08:46.293669 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 12:08:46.293790 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.65:40745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
2026-08-09T12:08:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2917 ms)
[----------] 6 tests from EncryptionPolicyTest (23722 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:08:46 dist-test-slave-sb2z krb5kdc[30050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:08:46 dist-test-slave-sb2z krb5kdc[30050](info): set up 2 sockets
Aug 09 12:08:46 dist-test-slave-sb2z krb5kdc[30050](info): commencing operation
krb5kdc: starting...
W20260809 12:08:47.687239 589 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:08:47 dist-test-slave-sb2z krb5kdc[30050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:08:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:08:48.022689 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:48.368806 30072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:48.369141 30072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:48.369316 30072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:48.376457 30072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:48.376600 30072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:48.376699 30072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:48.376812 30072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:48.376897 30072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:48.386374 30072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33617
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:48.387639 30072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:48.388952 30072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:48.400739 30077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:48.400722 30078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:48.400712 30082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:48.401822 30072 server_base.cc:1034] running on GCE node
I20260809 12:08:48.402554 30072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:48.403972 30072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:48.405192 30072 hybrid_clock.cc:648] HybridClock initialized: now 1786277328405146 us; error 44 us; skew 500 ppm
I20260809 12:08:48.405560 30072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:48 dist-test-slave-sb2z krb5kdc[30050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277328, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:48.411712 30072 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:08:48.413254 30072 webserver.cc:460] Webserver started at http://127.0.0.1:34579/ using document root <none> and password file <none>
I20260809 12:08:48.413933 30072 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:48.414091 30072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:48.414407 30072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:48.417498 30072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "4642abccdd504450b5d2ac69670980a4"
format_stamp: "Formatted at 2026-08-09 12:08:48 on dist-test-slave-sb2z"
server_key: "9392c2711e2257e42a1e083729a8704f"
server_key_iv: "636beda70a62f71f7a9bafd0ca6b3398"
server_key_version: "encryptionkey@0"
I20260809 12:08:48.418277 30072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "4642abccdd504450b5d2ac69670980a4"
format_stamp: "Formatted at 2026-08-09 12:08:48 on dist-test-slave-sb2z"
server_key: "9392c2711e2257e42a1e083729a8704f"
server_key_iv: "636beda70a62f71f7a9bafd0ca6b3398"
server_key_version: "encryptionkey@0"
I20260809 12:08:48.423372 30072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:08:48.426815 30096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:48.428197 30072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:08:48.428388 30072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "4642abccdd504450b5d2ac69670980a4"
format_stamp: "Formatted at 2026-08-09 12:08:48 on dist-test-slave-sb2z"
server_key: "9392c2711e2257e42a1e083729a8704f"
server_key_iv: "636beda70a62f71f7a9bafd0ca6b3398"
server_key_version: "encryptionkey@0"
I20260809 12:08:48.428592 30072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:48.443274 30072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:48.447711 30072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:48.448077 30072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:48.467304 30072 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33617
I20260809 12:08:48.467408 30169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33617 every 8 connection(s)
I20260809 12:08:48.468819 30072 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:48.472618 30170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:48.477958 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 30072
I20260809 12:08:48.478163 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:48.478976 589 external_mini_cluster.cc:1428] Setting key b9b8e85b34087dce0034221d03825a65
I20260809 12:08:48.484124 30170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4: Bootstrap starting.
I20260809 12:08:48.487980 30170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:48.488983 30170 log.cc:824] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:48.491921 30170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4: No bootstrap required, opened a new log
Aug 09 12:08:48 dist-test-slave-sb2z krb5kdc[30050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 12:08:48.500885 30170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4642abccdd504450b5d2ac69670980a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33617 } }
I20260809 12:08:48.501411 30170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:48.501539 30170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4642abccdd504450b5d2ac69670980a4, State: Initialized, Role: FOLLOWER
I20260809 12:08:48.502252 30170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4642abccdd504450b5d2ac69670980a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33617 } }
I20260809 12:08:48.502511 30170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:48.502678 30170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:48.502883 30170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:48.503477 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:48.481101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33645 (local address 127.0.0.1:33617)
0809 12:08:48.482014 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:48.482025 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:48.482071 (+ 46us) server_negotiation.cc:407] Connection header received
0809 12:08:48.483875 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:48.483898 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:48.484348 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:48.484682 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:48.486579 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:48.487999 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:48.489425 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:48.490145 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:48.490357 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:48.495197 (+ 4840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:48.495205 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:48.495222 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:48.495257 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:48.498568 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:48.499570 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:48.499584 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:48.499595 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:48.499778 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:48.500292 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:48.500296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:48.500298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:48.500834 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:48.501097 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:48.501785 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 12:08:48.502140 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":461,"threads_started":1}
I20260809 12:08:48.505393 30170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4642abccdd504450b5d2ac69670980a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33617 } }
I20260809 12:08:48.505967 30170 leader_election.cc:302] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4642abccdd504450b5d2ac69670980a4; no voters:
I20260809 12:08:48.506633 30170 leader_election.cc:288] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:48.506752 30177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:48.507934 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:08:48.508816 30170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:48.513819 30177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [term 1 LEADER]: Becoming Leader. State: Replica: 4642abccdd504450b5d2ac69670980a4, State: Running, Role: LEADER
I20260809 12:08:48.514539 30177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4642abccdd504450b5d2ac69670980a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33617 } }
I20260809 12:08:48.517848 30178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4642abccdd504450b5d2ac69670980a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4642abccdd504450b5d2ac69670980a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33617 } } }
I20260809 12:08:48.518374 30178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:48.518977 30180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4642abccdd504450b5d2ac69670980a4. Latest consensus state: current_term: 1 leader_uuid: "4642abccdd504450b5d2ac69670980a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4642abccdd504450b5d2ac69670980a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33617 } } }
I20260809 12:08:48.519352 30180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:48.528829 30196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:48.535501 30196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:48.549170 30196 catalog_manager.cc:1347] Generated new cluster ID: dfbd04e044e14ce6859cf44c777c9b60
I20260809 12:08:48.549365 30196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:48.552026 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:48.524932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49105 (local address 127.0.0.1:33617)
0809 12:08:48.525149 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:48.525159 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:48.532297 (+ 7138us) server_negotiation.cc:407] Connection header received
0809 12:08:48.532644 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:48.532656 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:48.532743 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:48.532818 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:48.535025 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:48.536261 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:48.538188 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:48.538924 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:48.539066 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:48.542632 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:48.542640 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:48.542645 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:48.542690 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:48.547764 (+ 5074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:48.548810 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:48.548815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:48.548818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:48.548975 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:48.549616 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:48.549621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:48.549624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:48.550067 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:48.550363 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:48.551483 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0809 12:08:48.551674 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 12:08:48.561834 30196 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:48.563678 30196 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:48.570730 30196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4642abccdd504450b5d2ac69670980a4: Generated new TSK 0
I20260809 12:08:48.571683 30196 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:08:48.581755 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30072
I20260809 12:08:48.583479 30165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:48.672573 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 30072
W20260809 12:08:48.697129 30047 connection.cc:535] client connection to 127.0.0.1:33617 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33617): Connection reset by peer (error 104)
2026-08-09T12:08:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2367 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:08:48 dist-test-slave-sb2z krb5kdc[30217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:08:48 dist-test-slave-sb2z krb5kdc[30217](info): set up 2 sockets
Aug 09 12:08:48 dist-test-slave-sb2z krb5kdc[30217](info): commencing operation
krb5kdc: starting...
W20260809 12:08:50.005358 589 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:08:50 dist-test-slave-sb2z krb5kdc[30217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:08:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:08:50.348322 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36557
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:50.691833 30241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:50.692103 30241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:50.692294 30241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:50.699669 30241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:50.699790 30241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:50.699884 30241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:50.699990 30241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:50.700068 30241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:50.709674 30241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:50.710897 30241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:50.712268 30241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:50.724119 30247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:50.724364 30250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:50.724457 30252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:50.725559 30241 server_base.cc:1034] running on GCE node
I20260809 12:08:50.726311 30241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:50.727689 30241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:50.728928 30241 hybrid_clock.cc:648] HybridClock initialized: now 1786277330728883 us; error 50 us; skew 500 ppm
I20260809 12:08:50.729291 30241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:50 dist-test-slave-sb2z krb5kdc[30217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277330, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:50.732743 30241 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:08:50.734221 30241 webserver.cc:460] Webserver started at http://127.0.0.1:43189/ using document root <none> and password file <none>
I20260809 12:08:50.734925 30241 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:50.735080 30241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:50.735409 30241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:50.738474 30241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "34e4863457b44dd9831c3378b13dd411"
format_stamp: "Formatted at 2026-08-09 12:08:50 on dist-test-slave-sb2z"
server_key: "780f2ddcd08fbd92095db57be89f912c"
server_key_iv: "845ff94b0269574efc8151e6aca0c753"
server_key_version: "encryptionkey@0"
I20260809 12:08:50.739266 30241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "34e4863457b44dd9831c3378b13dd411"
format_stamp: "Formatted at 2026-08-09 12:08:50 on dist-test-slave-sb2z"
server_key: "780f2ddcd08fbd92095db57be89f912c"
server_key_iv: "845ff94b0269574efc8151e6aca0c753"
server_key_version: "encryptionkey@0"
I20260809 12:08:50.744675 30241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:08:50.748176 30262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:50.749562 30241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:08:50.749773 30241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "34e4863457b44dd9831c3378b13dd411"
format_stamp: "Formatted at 2026-08-09 12:08:50 on dist-test-slave-sb2z"
server_key: "780f2ddcd08fbd92095db57be89f912c"
server_key_iv: "845ff94b0269574efc8151e6aca0c753"
server_key_version: "encryptionkey@0"
I20260809 12:08:50.749991 30241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:50.769640 30241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:50.774173 30241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:50.774470 30241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:50.793450 30241 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36557
I20260809 12:08:50.793494 30360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36557 every 8 connection(s)
I20260809 12:08:50.794827 30241 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:50.798827 30361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:50.801067 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 30241
I20260809 12:08:50.801281 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:50.801736 589 external_mini_cluster.cc:1428] Setting key 522507f6faa597b823779f51c2b5bb06
I20260809 12:08:50.810523 30361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411: Bootstrap starting.
I20260809 12:08:50.814200 30361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:50.815289 30361 log.cc:824] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:50.815395 30367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:50.803839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35191 (local address 127.0.0.1:36557)
0809 12:08:50.804674 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:50.804687 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:50.804738 (+ 51us) server_negotiation.cc:407] Connection header received
0809 12:08:50.806388 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:50.806409 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:50.806794 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:50.807119 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:50.807745 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:50.809079 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:50.811003 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:50.811715 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:50.811839 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:50.813169 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:50.813176 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:50.813189 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:50.813689 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:50.813762 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:50.813786 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:50.813898 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:50.814218 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 12:08:50.814406 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":380,"threads_started":1}
I20260809 12:08:50.818239 30361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411: No bootstrap required, opened a new log
I20260809 12:08:50.819159 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:08:50.826045 30361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e4863457b44dd9831c3378b13dd411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } }
I20260809 12:08:50.826419 30361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:50.826541 30361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e4863457b44dd9831c3378b13dd411, State: Initialized, Role: FOLLOWER
I20260809 12:08:50.827175 30361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e4863457b44dd9831c3378b13dd411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } }
I20260809 12:08:50.827438 30361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:50.827610 30361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:50.827803 30361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:50.830058 30367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:50.823649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42619 (local address 127.0.0.1:36557)
0809 12:08:50.823858 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:50.823869 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:50.824155 (+ 286us) server_negotiation.cc:407] Connection header received
0809 12:08:50.824363 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:50.824368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:50.824439 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:50.824555 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:50.825071 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:50.825806 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:50.827270 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:50.827977 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:50.828128 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:50.829291 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:50.829297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:50.829301 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:50.829328 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:50.829380 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:50.829395 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:50.829513 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:50.829720 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 12:08:50.829771 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:08:50.830065 30361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e4863457b44dd9831c3378b13dd411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } }
I20260809 12:08:50.830680 30361 leader_election.cc:302] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34e4863457b44dd9831c3378b13dd411; no voters:
I20260809 12:08:50.831261 30361 leader_election.cc:288] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:50.831400 30380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:50.832257 30380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [term 1 LEADER]: Becoming Leader. State: Replica: 34e4863457b44dd9831c3378b13dd411, State: Running, Role: LEADER
I20260809 12:08:50.832824 30380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e4863457b44dd9831c3378b13dd411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } }
I20260809 12:08:50.833499 30361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:50.835588 30383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34e4863457b44dd9831c3378b13dd411. Latest consensus state: current_term: 1 leader_uuid: "34e4863457b44dd9831c3378b13dd411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e4863457b44dd9831c3378b13dd411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } } }
I20260809 12:08:50.836054 30383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:50.836809 30381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34e4863457b44dd9831c3378b13dd411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e4863457b44dd9831c3378b13dd411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } } }
I20260809 12:08:50.837160 30381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:50.837730 30393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:50.844326 30393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:50.854465 30393 catalog_manager.cc:1347] Generated new cluster ID: 6d13fed5bded49c5b392c3e28077fc21
I20260809 12:08:50.854578 30393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:50.862812 30393 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:50.864030 30393 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:50.870592 30393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34e4863457b44dd9831c3378b13dd411: Generated new TSK 0
I20260809 12:08:50.871279 30393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:08:50.880285 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30241
W20260809 12:08:50.881405 30355 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35191
W20260809 12:08:50.882148 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30241: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:08:50.882344 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 30241
W20260809 12:08:50.907644 30215 connection.cc:535] client connection to 127.0.0.1:36557 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36557): Connection reset by peer (error 104)
2026-08-09T12:08:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:08:51 dist-test-slave-sb2z krb5kdc[30411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:08:51 dist-test-slave-sb2z krb5kdc[30411](info): set up 2 sockets
Aug 09 12:08:51 dist-test-slave-sb2z krb5kdc[30411](info): commencing operation
krb5kdc: starting...
W20260809 12:08:52.219805 589 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.005s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:08:52 dist-test-slave-sb2z krb5kdc[30411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:08:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:08:52.559333 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:52.907419 30427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:52.907724 30427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:52.907820 30427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:52.914927 30427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:52.915046 30427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:52.915103 30427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:52.915167 30427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:52.915206 30427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:52.924932 30427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:52.926231 30427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:52.927520 30427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:52.939371 30435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:52.939333 30438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:52.939409 30441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:52.940158 30427 server_base.cc:1034] running on GCE node
I20260809 12:08:52.940910 30427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:52.942234 30427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:52.943476 30427 hybrid_clock.cc:648] HybridClock initialized: now 1786277332943491 us; error 97 us; skew 500 ppm
I20260809 12:08:52.943890 30427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:52 dist-test-slave-sb2z krb5kdc[30411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277332, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:52.947108 30427 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:08:52.948520 30427 webserver.cc:460] Webserver started at http://127.0.0.1:43635/ using document root <none> and password file <none>
I20260809 12:08:52.949136 30427 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:52.949239 30427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:52.949479 30427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:52.952409 30427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "2e16f6ac3f374b2c9de4750527aacad7"
format_stamp: "Formatted at 2026-08-09 12:08:52 on dist-test-slave-sb2z"
server_key: "7c8b883f993fa831a8c0d955c65a781a"
server_key_iv: "a79edded9e3d6404df4c42e54a5fa59b"
server_key_version: "encryptionkey@0"
I20260809 12:08:52.953189 30427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "2e16f6ac3f374b2c9de4750527aacad7"
format_stamp: "Formatted at 2026-08-09 12:08:52 on dist-test-slave-sb2z"
server_key: "7c8b883f993fa831a8c0d955c65a781a"
server_key_iv: "a79edded9e3d6404df4c42e54a5fa59b"
server_key_version: "encryptionkey@0"
I20260809 12:08:52.958130 30427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:08:52.961464 30457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:52.962805 30427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:08:52.962982 30427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "2e16f6ac3f374b2c9de4750527aacad7"
format_stamp: "Formatted at 2026-08-09 12:08:52 on dist-test-slave-sb2z"
server_key: "7c8b883f993fa831a8c0d955c65a781a"
server_key_iv: "a79edded9e3d6404df4c42e54a5fa59b"
server_key_version: "encryptionkey@0"
I20260809 12:08:52.963150 30427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:52.968868 30427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:52.973286 30427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:52.973531 30427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:52.992851 30427 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42443
I20260809 12:08:52.992890 30533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42443 every 8 connection(s)
I20260809 12:08:52.994242 30427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:52.997836 30534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:53.003921 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 30427
I20260809 12:08:53.004140 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:53.004772 589 external_mini_cluster.cc:1428] Setting key 56a1a215b315821b82eaf37fec705230
I20260809 12:08:53.009104 30534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7: Bootstrap starting.
I20260809 12:08:53.012482 30534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:53.012913 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:53.006747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36777 (local address 127.0.0.1:42443)
0809 12:08:53.007652 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:53.007664 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:53.007703 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:08:53.009491 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:53.009517 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:53.009842 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:53.010221 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:53.011278 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:53.011284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:53.011296 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:53.011310 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:53.011370 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:53.011389 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:53.011657 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:53.011798 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 12:08:53.011975 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":365,"threads_started":1}
I20260809 12:08:53.013605 30534 log.cc:824] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:53.016682 30534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7: No bootstrap required, opened a new log
I20260809 12:08:53.016683 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:08:53.023730 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:53.021225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54429 (local address 127.0.0.1:42443)
0809 12:08:53.021416 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:53.021423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:53.021823 (+ 400us) server_negotiation.cc:407] Connection header received
0809 12:08:53.021949 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:53.021954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:53.022012 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:53.022127 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:53.023026 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:53.023034 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:53.023044 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:53.023062 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:53.023107 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:53.023132 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:53.023255 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:53.023326 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 12:08:53.023385 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:08:53.025082 30534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16f6ac3f374b2c9de4750527aacad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260809 12:08:53.025491 30534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:53.025583 30534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e16f6ac3f374b2c9de4750527aacad7, State: Initialized, Role: FOLLOWER
I20260809 12:08:53.026108 30534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16f6ac3f374b2c9de4750527aacad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260809 12:08:53.026319 30534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:53.026465 30534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:53.026604 30534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:53.029166 30534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16f6ac3f374b2c9de4750527aacad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260809 12:08:53.029621 30534 leader_election.cc:302] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e16f6ac3f374b2c9de4750527aacad7; no voters:
I20260809 12:08:53.030190 30534 leader_election.cc:288] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:53.030335 30546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:53.031517 30546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [term 1 LEADER]: Becoming Leader. State: Replica: 2e16f6ac3f374b2c9de4750527aacad7, State: Running, Role: LEADER
I20260809 12:08:53.032295 30546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16f6ac3f374b2c9de4750527aacad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260809 12:08:53.032433 30534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:53.035102 30548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e16f6ac3f374b2c9de4750527aacad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16f6ac3f374b2c9de4750527aacad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260809 12:08:53.035092 30549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e16f6ac3f374b2c9de4750527aacad7. Latest consensus state: current_term: 1 leader_uuid: "2e16f6ac3f374b2c9de4750527aacad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16f6ac3f374b2c9de4750527aacad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260809 12:08:53.035593 30549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:53.035662 30548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:53.037147 30555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:53.042661 30555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:53.051558 30555 catalog_manager.cc:1347] Generated new cluster ID: 4e4caa1039b2429ea15a13959aa7d99d
I20260809 12:08:53.051671 30555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:53.064131 30555 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:53.064555 30555 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:53.071624 30555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e16f6ac3f374b2c9de4750527aacad7: Generated new TSK 0
I20260809 12:08:53.072518 30555 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:08:53.087682 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30427
W20260809 12:08:53.089632 30527 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36777
W20260809 12:08:53.090437 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30427: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:08:53.090561 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 30427
2026-08-09T12:08:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2207 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:08:53 dist-test-slave-sb2z krb5kdc[30576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:08:53 dist-test-slave-sb2z krb5kdc[30576](info): set up 2 sockets
Aug 09 12:08:53 dist-test-slave-sb2z krb5kdc[30576](info): commencing operation
krb5kdc: starting...
W20260809 12:08:54.413550 589 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:08:54 dist-test-slave-sb2z krb5kdc[30576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:08:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:08:54.758252 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46255
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:55.108808 30596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:55.109102 30596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:55.109205 30596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:55.116513 30596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:55.116621 30596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:55.116678 30596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:55.116747 30596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:55.116794 30596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:55.126866 30596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:55.128139 30596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:55.129362 30596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:55.141484 30603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:55.141436 30604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:55.141506 30606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:55.143087 30596 server_base.cc:1034] running on GCE node
I20260809 12:08:55.143929 30596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:55.145272 30596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:55.146472 30596 hybrid_clock.cc:648] HybridClock initialized: now 1786277335146433 us; error 34 us; skew 500 ppm
I20260809 12:08:55.146819 30596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:55 dist-test-slave-sb2z krb5kdc[30576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277335, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:55.153913 30596 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:08:55.155500 30596 webserver.cc:460] Webserver started at http://127.0.0.1:41759/ using document root <none> and password file <none>
I20260809 12:08:55.156237 30596 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:55.156344 30596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:55.156605 30596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:55.159747 30596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "75396a61bbaa4c40980a19324f011376"
format_stamp: "Formatted at 2026-08-09 12:08:55 on dist-test-slave-sb2z"
server_key: "fec80d9874c4c36cd644397547248034"
server_key_iv: "1c4961166baf7aa90db192782c8cf32e"
server_key_version: "encryptionkey@0"
I20260809 12:08:55.160466 30596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "75396a61bbaa4c40980a19324f011376"
format_stamp: "Formatted at 2026-08-09 12:08:55 on dist-test-slave-sb2z"
server_key: "fec80d9874c4c36cd644397547248034"
server_key_iv: "1c4961166baf7aa90db192782c8cf32e"
server_key_version: "encryptionkey@0"
I20260809 12:08:55.165482 30596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:08:55.168830 30616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:55.170025 30596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:08:55.170204 30596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "75396a61bbaa4c40980a19324f011376"
format_stamp: "Formatted at 2026-08-09 12:08:55 on dist-test-slave-sb2z"
server_key: "fec80d9874c4c36cd644397547248034"
server_key_iv: "1c4961166baf7aa90db192782c8cf32e"
server_key_version: "encryptionkey@0"
I20260809 12:08:55.170394 30596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:55.185925 30596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:55.189827 30596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:55.190124 30596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:55.209067 30596 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46255
I20260809 12:08:55.209102 30695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46255 every 8 connection(s)
I20260809 12:08:55.210429 30596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:55.211730 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 30596
I20260809 12:08:55.211928 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:55.212476 589 external_mini_cluster.cc:1428] Setting key d4e227b25eeee946fc6e135f6d0eaa1e
I20260809 12:08:55.215075 30696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:08:55 dist-test-slave-sb2z krb5kdc[30576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 12:08:55.228327 30696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376: Bootstrap starting.
I20260809 12:08:55.231413 30696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:55.232436 30696 log.cc:824] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:55.235306 30696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376: No bootstrap required, opened a new log
I20260809 12:08:55.235628 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:55.214271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44897 (local address 127.0.0.1:46255)
0809 12:08:55.215064 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:55.215075 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:55.215141 (+ 66us) server_negotiation.cc:407] Connection header received
0809 12:08:55.216869 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:55.216889 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:55.217245 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:55.217584 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:55.219466 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:55.220893 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:55.222758 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:55.223388 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:55.223606 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:55.228418 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:55.228425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:55.228440 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:55.228508 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:55.231579 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:55.232267 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:55.232283 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:55.232288 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:55.232452 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:55.232851 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:55.232857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:55.232860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:55.233410 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:55.233738 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:55.234187 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 12:08:55.234601 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":375,"threads_started":1}
I20260809 12:08:55.239156 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:08:55.243080 30696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75396a61bbaa4c40980a19324f011376" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46255 } }
I20260809 12:08:55.243471 30696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:55.243623 30696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75396a61bbaa4c40980a19324f011376, State: Initialized, Role: FOLLOWER
I20260809 12:08:55.244390 30696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75396a61bbaa4c40980a19324f011376" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46255 } }
I20260809 12:08:55.244689 30696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:55.244858 30696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:55.245031 30696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:55.247977 30696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75396a61bbaa4c40980a19324f011376" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46255 } }
I20260809 12:08:55.248580 30696 leader_election.cc:302] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75396a61bbaa4c40980a19324f011376; no voters:
I20260809 12:08:55.249239 30696 leader_election.cc:288] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:55.249387 30712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:55.250474 30712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [term 1 LEADER]: Becoming Leader. State: Replica: 75396a61bbaa4c40980a19324f011376, State: Running, Role: LEADER
I20260809 12:08:55.251192 30712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75396a61bbaa4c40980a19324f011376" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46255 } }
I20260809 12:08:55.251505 30696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:55.254456 30713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75396a61bbaa4c40980a19324f011376" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75396a61bbaa4c40980a19324f011376" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46255 } } }
I20260809 12:08:55.254848 30713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:55.255434 30716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75396a61bbaa4c40980a19324f011376. Latest consensus state: current_term: 1 leader_uuid: "75396a61bbaa4c40980a19324f011376" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75396a61bbaa4c40980a19324f011376" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46255 } } }
I20260809 12:08:55.255791 30716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:55.263142 30724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:55.263221 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:55.243669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52505 (local address 127.0.0.1:46255)
0809 12:08:55.243887 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:55.243894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:55.244268 (+ 374us) server_negotiation.cc:407] Connection header received
0809 12:08:55.244419 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:55.244423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:55.244494 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:55.244568 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:55.246507 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:55.247219 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:55.249064 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:55.249928 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:55.250121 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:55.253529 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:55.253534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:55.253539 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:08:55.253581 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:55.258084 (+ 4503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:55.259226 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:55.259234 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:55.259240 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:55.259714 (+ 474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:08:55.260304 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:08:55.260311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:08:55.260317 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:08:55.260777 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:55.262424 (+ 1647us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:55.262666 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 12:08:55.262940 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:08:55.268132 30724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:55.287092 30724 catalog_manager.cc:1347] Generated new cluster ID: fa22cb0e0cb942099d081d7ddf71697b
I20260809 12:08:55.287209 30724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:55.327728 30724 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:55.329015 30724 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:55.334981 30724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75396a61bbaa4c40980a19324f011376: Generated new TSK 0
I20260809 12:08:55.336057 30724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:08:55.355566 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30596
I20260809 12:08:55.356604 30688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:08:55.452273 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 30596
2026-08-09T12:08:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2363 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:08:55 dist-test-slave-sb2z krb5kdc[30747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:08:55 dist-test-slave-sb2z krb5kdc[30747](info): set up 2 sockets
Aug 09 12:08:55 dist-test-slave-sb2z krb5kdc[30747](info): commencing operation
krb5kdc: starting...
W20260809 12:08:56.793722 589 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:08:57 dist-test-slave-sb2z krb5kdc[30747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:08:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:08:57.142592 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:57.496718 30763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:57.497213 30763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:57.497395 30763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:57.508137 30763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:57.508315 30763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:57.508402 30763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:57.508496 30763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:57.508574 30763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:57.524590 30763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:57.526021 30763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:57.527340 30763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:57.539358 30769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:57.539323 30771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:57.539552 30773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:57.540203 30763 server_base.cc:1034] running on GCE node
I20260809 12:08:57.541024 30763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:57.542361 30763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:57.543644 30763 hybrid_clock.cc:648] HybridClock initialized: now 1786277337543609 us; error 49 us; skew 500 ppm
I20260809 12:08:57.543998 30763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:57 dist-test-slave-sb2z krb5kdc[30747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277337, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:57.547034 30763 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:08:57.548497 30763 webserver.cc:460] Webserver started at http://127.0.0.1:44215/ using document root <none> and password file <none>
I20260809 12:08:57.549201 30763 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:57.549350 30763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:57.549670 30763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:57.552683 30763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "68572b8927254884a09356b8db5da012"
format_stamp: "Formatted at 2026-08-09 12:08:57 on dist-test-slave-sb2z"
server_key: "199b6e1e9662a13b52e8611a601d18db"
server_key_iv: "761705a439c16559682320f234fafd5c"
server_key_version: "encryptionkey@0"
I20260809 12:08:57.553450 30763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "68572b8927254884a09356b8db5da012"
format_stamp: "Formatted at 2026-08-09 12:08:57 on dist-test-slave-sb2z"
server_key: "199b6e1e9662a13b52e8611a601d18db"
server_key_iv: "761705a439c16559682320f234fafd5c"
server_key_version: "encryptionkey@0"
I20260809 12:08:57.558566 30763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 12:08:57.561980 30782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:57.563242 30763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:08:57.563421 30763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "68572b8927254884a09356b8db5da012"
format_stamp: "Formatted at 2026-08-09 12:08:57 on dist-test-slave-sb2z"
server_key: "199b6e1e9662a13b52e8611a601d18db"
server_key_iv: "761705a439c16559682320f234fafd5c"
server_key_version: "encryptionkey@0"
I20260809 12:08:57.563684 30763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:57.575654 30763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:57.579546 30763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:57.579838 30763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:57.598645 30763 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43185
I20260809 12:08:57.598686 30861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43185 every 8 connection(s)
I20260809 12:08:57.600086 30763 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:57.603856 30862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:57.607261 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 30763
I20260809 12:08:57.607450 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:57.608007 589 external_mini_cluster.cc:1428] Setting key 33b14434bc488b1178c24b304a3732f1
I20260809 12:08:57.616219 30862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012: Bootstrap starting.
I20260809 12:08:57.619834 30862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:57.620932 30862 log.cc:824] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:57.621253 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:57.609832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53711 (local address 127.0.0.1:43185)
0809 12:08:57.610605 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:57.610616 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:57.610670 (+ 54us) server_negotiation.cc:407] Connection header received
0809 12:08:57.612382 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:57.612404 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:57.612794 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:57.613106 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:57.613687 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:57.615001 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:57.616870 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:57.617494 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:57.617629 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:57.618893 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:57.618900 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:57.618912 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:57.619415 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:57.619484 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:57.619510 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:57.619651 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:57.619928 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 12:08:57.620241 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":386,"threads_started":1}
I20260809 12:08:57.625026 30862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012: No bootstrap required, opened a new log
I20260809 12:08:57.625098 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:08:57.635133 30862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68572b8927254884a09356b8db5da012" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43185 } }
I20260809 12:08:57.635489 30862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:57.635692 30862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68572b8927254884a09356b8db5da012, State: Initialized, Role: FOLLOWER
I20260809 12:08:57.636361 30862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68572b8927254884a09356b8db5da012" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43185 } }
I20260809 12:08:57.636677 30862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:57.636883 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:57.630261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52939 (local address 127.0.0.1:43185)
0809 12:08:57.630452 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:57.630458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:57.630846 (+ 388us) server_negotiation.cc:407] Connection header received
0809 12:08:57.631045 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:57.631050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:57.631138 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:57.631246 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:57.631797 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:57.632910 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:57.634207 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:08:57.634824 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:08:57.634971 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:08:57.636050 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:57.636056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:57.636060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:57.636080 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:57.636150 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:57.636169 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:57.636267 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:57.636416 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 12:08:57.636504 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:08:57.636842 30862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:57.637066 30862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:57.639333 30862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68572b8927254884a09356b8db5da012" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43185 } }
I20260809 12:08:57.639844 30862 leader_election.cc:302] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68572b8927254884a09356b8db5da012; no voters:
I20260809 12:08:57.640399 30862 leader_election.cc:288] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:57.640545 30877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:57.641783 30877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [term 1 LEADER]: Becoming Leader. State: Replica: 68572b8927254884a09356b8db5da012, State: Running, Role: LEADER
I20260809 12:08:57.642347 30877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68572b8927254884a09356b8db5da012" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43185 } }
I20260809 12:08:57.642606 30862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:57.645102 30881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68572b8927254884a09356b8db5da012. Latest consensus state: current_term: 1 leader_uuid: "68572b8927254884a09356b8db5da012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68572b8927254884a09356b8db5da012" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43185 } } }
I20260809 12:08:57.645406 30880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68572b8927254884a09356b8db5da012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68572b8927254884a09356b8db5da012" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43185 } } }
I20260809 12:08:57.645641 30881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:57.645716 30880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:57.648357 30888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:57.653560 30888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:57.662626 30888 catalog_manager.cc:1347] Generated new cluster ID: dcc2dd60336c42c9990d176b3eb574a9
I20260809 12:08:57.662732 30888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:57.671543 30888 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:57.672658 30888 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:57.681572 30888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68572b8927254884a09356b8db5da012: Generated new TSK 0
I20260809 12:08:57.682376 30888 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:08:57.687865 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30763
W20260809 12:08:57.689045 30854 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53711
W20260809 12:08:57.689733 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30763: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:08:57.689882 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 30763
2026-08-09T12:08:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2234 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:08:57 dist-test-slave-sb2z krb5kdc[30906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:08:57 dist-test-slave-sb2z krb5kdc[30906](info): set up 2 sockets
Aug 09 12:08:57 dist-test-slave-sb2z krb5kdc[30906](info): commencing operation
krb5kdc: starting...
W20260809 12:08:59.003362 589 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:08:59 dist-test-slave-sb2z krb5kdc[30906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:08:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:08:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:08:59.348001 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37895
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:08:59.689316 30926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:08:59.689620 30926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:08:59.689821 30926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:08:59.697116 30926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:08:59.697247 30926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:08:59.697353 30926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:08:59.697440 30926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:08:59.697480 30926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:08:59.707396 30926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:08:59.708685 30926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:08:59.709972 30926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:08:59.721925 30941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:59.722038 30937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:08:59.722028 30938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:08:59.723122 30926 server_base.cc:1034] running on GCE node
I20260809 12:08:59.724025 30926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:08:59.725479 30926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:08:59.726729 30926 hybrid_clock.cc:648] HybridClock initialized: now 1786277339726670 us; error 46 us; skew 500 ppm
I20260809 12:08:59.727125 30926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:08:59 dist-test-slave-sb2z krb5kdc[30906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277339, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:08:59.730515 30926 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:08:59.732191 30926 webserver.cc:460] Webserver started at http://127.0.0.1:37019/ using document root <none> and password file <none>
I20260809 12:08:59.732880 30926 fs_manager.cc:359] Metadata directory not provided
I20260809 12:08:59.733031 30926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:08:59.733355 30926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:08:59.736402 30926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "721da7e5ba5b451c8de0ba5c1b262ed1"
format_stamp: "Formatted at 2026-08-09 12:08:59 on dist-test-slave-sb2z"
server_key: "6c18e48ea6c19561c5cda1481f135e21"
server_key_iv: "0ebbb5499ae714483e582824020ec7e9"
server_key_version: "encryptionkey@0"
I20260809 12:08:59.737188 30926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "721da7e5ba5b451c8de0ba5c1b262ed1"
format_stamp: "Formatted at 2026-08-09 12:08:59 on dist-test-slave-sb2z"
server_key: "6c18e48ea6c19561c5cda1481f135e21"
server_key_iv: "0ebbb5499ae714483e582824020ec7e9"
server_key_version: "encryptionkey@0"
I20260809 12:08:59.742431 30926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260809 12:08:59.745952 30950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:08:59.747277 30926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:08:59.747474 30926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "721da7e5ba5b451c8de0ba5c1b262ed1"
format_stamp: "Formatted at 2026-08-09 12:08:59 on dist-test-slave-sb2z"
server_key: "6c18e48ea6c19561c5cda1481f135e21"
server_key_iv: "0ebbb5499ae714483e582824020ec7e9"
server_key_version: "encryptionkey@0"
I20260809 12:08:59.747764 30926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:08:59.754014 30926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:08:59.758236 30926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:08:59.758533 30926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:08:59.777882 30926 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37895
I20260809 12:08:59.777973 31025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37895 every 8 connection(s)
I20260809 12:08:59.779332 30926 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:08:59.783268 31027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:08:59.789305 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 30926
I20260809 12:08:59.789518 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:08:59.789954 589 external_mini_cluster.cc:1428] Setting key 4632cea48cebbf4befe78b623539740b
I20260809 12:08:59.794250 31027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1: Bootstrap starting.
I20260809 12:08:59.797652 31027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1: Neither blocks nor log segments found. Creating new log.
I20260809 12:08:59.797937 31031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:59.791883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38393 (local address 127.0.0.1:37895)
0809 12:08:59.792777 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:59.792788 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:59.792828 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:08:59.794585 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:59.794607 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:59.794919 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:59.795292 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:59.796379 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:59.796385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:59.796397 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:59.796410 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:59.796463 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:59.796481 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:59.796717 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:59.796853 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 12:08:59.797029 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":417,"threads_started":1}
I20260809 12:08:59.798764 31027 log.cc:824] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1: Log is configured to *not* fsync() on all Append() calls
I20260809 12:08:59.801852 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:08:59.801865 31027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1: No bootstrap required, opened a new log
I20260809 12:08:59.808816 31031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:08:59.806339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38001 (local address 127.0.0.1:37895)
0809 12:08:59.806530 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:08:59.806542 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:08:59.806877 (+ 335us) server_negotiation.cc:407] Connection header received
0809 12:08:59.807030 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:08:59.807033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:08:59.807081 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:08:59.807163 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:08:59.808157 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:08:59.808161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:08:59.808164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:08:59.808177 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:08:59.808214 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:08:59.808227 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:08:59.808319 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 12:08:59.808445 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 12:08:59.808491 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 12:08:59.810734 31027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721da7e5ba5b451c8de0ba5c1b262ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37895 } }
I20260809 12:08:59.811105 31027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:08:59.811219 31027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721da7e5ba5b451c8de0ba5c1b262ed1, State: Initialized, Role: FOLLOWER
I20260809 12:08:59.811791 31027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721da7e5ba5b451c8de0ba5c1b262ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37895 } }
I20260809 12:08:59.812052 31027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:08:59.812198 31027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:08:59.812376 31027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:08:59.814589 31027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721da7e5ba5b451c8de0ba5c1b262ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37895 } }
I20260809 12:08:59.815064 31027 leader_election.cc:302] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 721da7e5ba5b451c8de0ba5c1b262ed1; no voters:
I20260809 12:08:59.815750 31027 leader_election.cc:288] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:08:59.815811 31044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:08:59.816716 31044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [term 1 LEADER]: Becoming Leader. State: Replica: 721da7e5ba5b451c8de0ba5c1b262ed1, State: Running, Role: LEADER
I20260809 12:08:59.817345 31044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721da7e5ba5b451c8de0ba5c1b262ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37895 } }
I20260809 12:08:59.818588 31027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:08:59.820231 31045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "721da7e5ba5b451c8de0ba5c1b262ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721da7e5ba5b451c8de0ba5c1b262ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37895 } } }
I20260809 12:08:59.820142 31046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 721da7e5ba5b451c8de0ba5c1b262ed1. Latest consensus state: current_term: 1 leader_uuid: "721da7e5ba5b451c8de0ba5c1b262ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721da7e5ba5b451c8de0ba5c1b262ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37895 } } }
I20260809 12:08:59.820602 31045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:59.820600 31046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:08:59.823675 31053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:08:59.828531 31053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:08:59.840730 31053 catalog_manager.cc:1347] Generated new cluster ID: ea06892f50054005953d4b915d4aa763
I20260809 12:08:59.840858 31053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:08:59.862138 31053 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:08:59.862569 31053 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:08:59.869822 31053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 721da7e5ba5b451c8de0ba5c1b262ed1: Generated new TSK 0
I20260809 12:08:59.870568 31053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:08:59.881026 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30926
W20260809 12:08:59.882112 31019 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38393
W20260809 12:08:59.882789 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 30926: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:08:59.882943 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 30926
2026-08-09T12:08:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2200 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:00 dist-test-slave-sb2z krb5kdc[31078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:00 dist-test-slave-sb2z krb5kdc[31078](info): set up 2 sockets
Aug 09 12:09:00 dist-test-slave-sb2z krb5kdc[31078](info): commencing operation
krb5kdc: starting...
W20260809 12:09:01.220870 589 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.003s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:01 dist-test-slave-sb2z krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:09:01.558632 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:01.906374 31098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:01.906651 31098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:01.906788 31098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:01.914059 31098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:01.914166 31098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:01.914232 31098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:01.914356 31098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:01.914458 31098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:01.924113 31098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:01.925336 31098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:01.926615 31098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:01.938246 31105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:01.938349 31106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:01.938352 31109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:01.939455 31098 server_base.cc:1034] running on GCE node
I20260809 12:09:01.940236 31098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:01.941614 31098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:01.942852 31098 hybrid_clock.cc:648] HybridClock initialized: now 1786277341942818 us; error 50 us; skew 500 ppm
I20260809 12:09:01.943210 31098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:01 dist-test-slave-sb2z krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277341, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:01.949242 31098 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:09:01.950762 31098 webserver.cc:460] Webserver started at http://127.0.0.1:39181/ using document root <none> and password file <none>
I20260809 12:09:01.951447 31098 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:01.951625 31098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:01.951946 31098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:01.954950 31098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "2617d236dd154552b214d05d4edfc8fd"
format_stamp: "Formatted at 2026-08-09 12:09:01 on dist-test-slave-sb2z"
server_key: "41fc13d78e58cfb9387f64662a934b0f"
server_key_iv: "6e71245d385ef11d49eed24e0b49f74d"
server_key_version: "encryptionkey@0"
I20260809 12:09:01.955830 31098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "2617d236dd154552b214d05d4edfc8fd"
format_stamp: "Formatted at 2026-08-09 12:09:01 on dist-test-slave-sb2z"
server_key: "41fc13d78e58cfb9387f64662a934b0f"
server_key_iv: "6e71245d385ef11d49eed24e0b49f74d"
server_key_version: "encryptionkey@0"
I20260809 12:09:01.960911 31098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:09:01.964340 31119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:01.965685 31098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:09:01.965881 31098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "2617d236dd154552b214d05d4edfc8fd"
format_stamp: "Formatted at 2026-08-09 12:09:01 on dist-test-slave-sb2z"
server_key: "41fc13d78e58cfb9387f64662a934b0f"
server_key_iv: "6e71245d385ef11d49eed24e0b49f74d"
server_key_version: "encryptionkey@0"
I20260809 12:09:01.966090 31098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:01.998608 31098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:02.002660 31098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:02.003000 31098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:02.022323 31098 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42107
I20260809 12:09:02.022370 31198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42107 every 8 connection(s)
I20260809 12:09:02.023813 31098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:02.027593 31199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:02.033856 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 31098
I20260809 12:09:02.034082 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:02.034499 589 external_mini_cluster.cc:1428] Setting key 6bd639fda472e59312554e4c00b96125
I20260809 12:09:02.039237 31199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd: Bootstrap starting.
I20260809 12:09:02.043125 31199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:02.044435 31199 log.cc:824] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:02.047749 31199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd: No bootstrap required, opened a new log
Aug 09 12:09:02 dist-test-slave-sb2z krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 12:09:02.058194 31199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2617d236dd154552b214d05d4edfc8fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42107 } }
I20260809 12:09:02.058696 31199 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:02.058840 31199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2617d236dd154552b214d05d4edfc8fd, State: Initialized, Role: FOLLOWER
I20260809 12:09:02.059691 31199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2617d236dd154552b214d05d4edfc8fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42107 } }
I20260809 12:09:02.059999 31199 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:02.060159 31199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:02.060352 31199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:02.060814 31205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:02.036634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:54241 (local address 127.0.0.1:42107)
0809 12:09:02.037428 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:02.037443 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:02.037490 (+ 47us) server_negotiation.cc:407] Connection header received
0809 12:09:02.039920 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:02.039949 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:02.040429 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:02.040877 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:02.042203 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:02.044273 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:02.045978 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:02.047005 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:02.047207 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:02.051491 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:02.051502 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:02.051552 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:02.051596 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:02.055936 (+ 4340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:02.056764 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:02.056773 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:02.056783 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:02.057005 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:02.057440 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:02.057447 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:02.057452 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:02.058205 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:02.058537 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:02.058995 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 12:09:02.059345 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":347,"threads_started":1}
I20260809 12:09:02.062953 31199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2617d236dd154552b214d05d4edfc8fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42107 } }
I20260809 12:09:02.063493 31199 leader_election.cc:302] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2617d236dd154552b214d05d4edfc8fd; no voters:
I20260809 12:09:02.064339 31199 leader_election.cc:288] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:02.064459 31207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:02.065016 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:02.065905 31207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [term 1 LEADER]: Becoming Leader. State: Replica: 2617d236dd154552b214d05d4edfc8fd, State: Running, Role: LEADER
I20260809 12:09:02.066644 31199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:02.066565 31207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2617d236dd154552b214d05d4edfc8fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42107 } }
I20260809 12:09:02.069144 31210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2617d236dd154552b214d05d4edfc8fd. Latest consensus state: current_term: 1 leader_uuid: "2617d236dd154552b214d05d4edfc8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2617d236dd154552b214d05d4edfc8fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42107 } } }
I20260809 12:09:02.069552 31210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:02.070122 31209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2617d236dd154552b214d05d4edfc8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2617d236dd154552b214d05d4edfc8fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42107 } } }
I20260809 12:09:02.070417 31209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:02.076359 31216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:02.083860 31216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:02.096411 31216 catalog_manager.cc:1347] Generated new cluster ID: 4f610b0162694c41967704807bdfce83
I20260809 12:09:02.096539 31216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:02.099648 31205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:02.080575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:48563 (local address 127.0.0.1:42107)
0809 12:09:02.080787 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:02.080797 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:02.081175 (+ 378us) server_negotiation.cc:407] Connection header received
0809 12:09:02.081472 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:02.081478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:02.081562 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:02.081646 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:02.083836 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:02.085111 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:02.087078 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:02.087815 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:02.087921 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:02.091409 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:02.091416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:02.091421 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:02.091469 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:02.095311 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:02.096399 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:02.096405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:02.096408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:02.096571 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:02.097164 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:02.097170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:02.097173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:02.097600 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:02.097871 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:02.099045 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0809 12:09:02.099241 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 12:09:02.119297 31216 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:02.120971 31216 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:02.131465 31216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2617d236dd154552b214d05d4edfc8fd: Generated new TSK 0
I20260809 12:09:02.132373 31216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:02.139367 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31098
I20260809 12:09:02.140564 31191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:02.232547 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 31098
W20260809 12:09:02.259657 31073 connection.cc:535] client connection to 127.0.0.1:42107 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42107): Connection reset by peer (error 104)
2026-08-09T12:09:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2345 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:02 dist-test-slave-sb2z krb5kdc[31250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:02 dist-test-slave-sb2z krb5kdc[31250](info): set up 2 sockets
Aug 09 12:09:02 dist-test-slave-sb2z krb5kdc[31250](info): commencing operation
krb5kdc: starting...
W20260809 12:09:03.550402 589 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:03 dist-test-slave-sb2z krb5kdc[31250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:09:03.896503 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:04.238242 31271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:04.238507 31271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:04.238685 31271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:04.245693 31271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:04.245792 31271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:04.245921 31271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:04.246029 31271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:04.246114 31271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:04.255574 31271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:04.256803 31271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:04.258102 31271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:04.270011 31276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:04.270063 31281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:04.270110 31278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:04.271080 31271 server_base.cc:1034] running on GCE node
I20260809 12:09:04.271925 31271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:04.273332 31271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:04.274528 31271 hybrid_clock.cc:648] HybridClock initialized: now 1786277344274474 us; error 60 us; skew 500 ppm
I20260809 12:09:04.274885 31271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:04 dist-test-slave-sb2z krb5kdc[31250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277344, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:04.278093 31271 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:09:04.279742 31271 webserver.cc:460] Webserver started at http://127.0.0.1:44171/ using document root <none> and password file <none>
I20260809 12:09:04.280413 31271 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:04.280548 31271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:04.280874 31271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:04.283895 31271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "933b18cfe57e43849db724fb9f5587f5"
format_stamp: "Formatted at 2026-08-09 12:09:04 on dist-test-slave-sb2z"
server_key: "fc89e026bc10c36b13bf0e4407a64197"
server_key_iv: "1cab769c1793a9ec59761e1a8c4eec37"
server_key_version: "encryptionkey@0"
I20260809 12:09:04.284658 31271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "933b18cfe57e43849db724fb9f5587f5"
format_stamp: "Formatted at 2026-08-09 12:09:04 on dist-test-slave-sb2z"
server_key: "fc89e026bc10c36b13bf0e4407a64197"
server_key_iv: "1cab769c1793a9ec59761e1a8c4eec37"
server_key_version: "encryptionkey@0"
I20260809 12:09:04.289640 31271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:09:04.292969 31291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:04.294220 31271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:09:04.294401 31271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "933b18cfe57e43849db724fb9f5587f5"
format_stamp: "Formatted at 2026-08-09 12:09:04 on dist-test-slave-sb2z"
server_key: "fc89e026bc10c36b13bf0e4407a64197"
server_key_iv: "1cab769c1793a9ec59761e1a8c4eec37"
server_key_version: "encryptionkey@0"
I20260809 12:09:04.294585 31271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:04.308851 31271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:04.312670 31271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:04.312943 31271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:04.332054 31271 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45055
I20260809 12:09:04.332161 31367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45055 every 8 connection(s)
I20260809 12:09:04.333387 31271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:04.336973 31370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:04.337895 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 31271
I20260809 12:09:04.338107 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:04.338575 589 external_mini_cluster.cc:1428] Setting key d6a3ca0c963ae9413995246e2d8c6bbd
I20260809 12:09:04.351758 31370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5: Bootstrap starting.
I20260809 12:09:04.353720 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:04.340467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:52217 (local address 127.0.0.1:45055)
0809 12:09:04.341322 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:04.341338 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:04.341397 (+ 59us) server_negotiation.cc:407] Connection header received
0809 12:09:04.343732 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:04.343760 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:04.344277 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:04.344717 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:04.345207 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:04.347092 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:04.348604 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:04.349672 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:04.349839 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:04.351014 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:04.351026 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:04.351043 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:04.351769 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:04.351866 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:04.351899 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:04.352084 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:04.352295 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 12:09:04.352455 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":327,"threads_started":1}
I20260809 12:09:04.355367 31370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:04.356830 31370 log.cc:824] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:04.357733 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:04.360073 31370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5: No bootstrap required, opened a new log
I20260809 12:09:04.369680 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:04.362963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:51463 (local address 127.0.0.1:45055)
0809 12:09:04.363148 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:04.363154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:04.363698 (+ 544us) server_negotiation.cc:407] Connection header received
0809 12:09:04.363823 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:04.363828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:04.363911 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:04.364028 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:04.364477 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:04.365627 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:04.366912 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:04.367637 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:04.367774 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:04.368874 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:04.368879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:04.368882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:04.368902 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:04.368962 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:04.368977 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:04.369061 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:04.369292 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 12:09:04.369360 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:09:04.370118 31370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933b18cfe57e43849db724fb9f5587f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } }
I20260809 12:09:04.370507 31370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:04.370631 31370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933b18cfe57e43849db724fb9f5587f5, State: Initialized, Role: FOLLOWER
I20260809 12:09:04.371260 31370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933b18cfe57e43849db724fb9f5587f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } }
I20260809 12:09:04.371556 31370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:04.371726 31370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:04.371920 31370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:04.374177 31370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933b18cfe57e43849db724fb9f5587f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } }
I20260809 12:09:04.374675 31370 leader_election.cc:302] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 933b18cfe57e43849db724fb9f5587f5; no voters:
I20260809 12:09:04.375336 31370 leader_election.cc:288] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:04.375437 31382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:04.376175 31382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [term 1 LEADER]: Becoming Leader. State: Replica: 933b18cfe57e43849db724fb9f5587f5, State: Running, Role: LEADER
I20260809 12:09:04.376788 31382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933b18cfe57e43849db724fb9f5587f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } }
I20260809 12:09:04.377635 31370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:04.380215 31384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 933b18cfe57e43849db724fb9f5587f5. Latest consensus state: current_term: 1 leader_uuid: "933b18cfe57e43849db724fb9f5587f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933b18cfe57e43849db724fb9f5587f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } } }
I20260809 12:09:04.380298 31383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "933b18cfe57e43849db724fb9f5587f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933b18cfe57e43849db724fb9f5587f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } } }
I20260809 12:09:04.380713 31383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:04.380714 31384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:04.381644 31394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:04.395270 31394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:04.403401 31394 catalog_manager.cc:1347] Generated new cluster ID: 6cdc2bc4c0c649429e537445e1dd63df
I20260809 12:09:04.403592 31394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:04.425402 31394 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:04.427326 31394 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:04.442116 31394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 933b18cfe57e43849db724fb9f5587f5: Generated new TSK 0
I20260809 12:09:04.443204 31394 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:04.456216 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31271
W20260809 12:09:04.457451 31361 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.157:52217
W20260809 12:09:04.458225 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31271: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:09:04.458380 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 31271
W20260809 12:09:04.483492 31248 connection.cc:535] client connection to 127.0.0.1:45055 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45055): Connection reset by peer (error 104)
2026-08-09T12:09:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2224 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:04 dist-test-slave-sb2z krb5kdc[31410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:04 dist-test-slave-sb2z krb5kdc[31410](info): set up 2 sockets
Aug 09 12:09:04 dist-test-slave-sb2z krb5kdc[31410](info): commencing operation
krb5kdc: starting...
W20260809 12:09:05.779937 589 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.004s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:06 dist-test-slave-sb2z krb5kdc[31410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:09:06.126238 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36145
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:06.466907 31430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:06.467207 31430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:06.467305 31430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:06.474558 31430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:06.474664 31430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:06.474718 31430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:06.474781 31430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:06.474823 31430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:06.484511 31430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:06.485661 31430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:06.486881 31430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:06.498606 31441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:06.498514 31442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:06.498569 31446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:06.499575 31430 server_base.cc:1034] running on GCE node
I20260809 12:09:06.500403 31430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:06.501833 31430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:06.503072 31430 hybrid_clock.cc:648] HybridClock initialized: now 1786277346503030 us; error 66 us; skew 500 ppm
I20260809 12:09:06.503415 31430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:06 dist-test-slave-sb2z krb5kdc[31410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277346, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:06.506609 31430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:09:06.508170 31430 webserver.cc:460] Webserver started at http://127.0.0.1:41495/ using document root <none> and password file <none>
I20260809 12:09:06.508849 31430 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:06.508999 31430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:06.509311 31430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:06.512290 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "f86445c074dc49e69f11336aba5db7d3"
format_stamp: "Formatted at 2026-08-09 12:09:06 on dist-test-slave-sb2z"
server_key: "8dbbf11074b89bf0e2912af3d0315f4f"
server_key_iv: "d8603fbc28b6607f9c6338a3f14b4215"
server_key_version: "encryptionkey@0"
I20260809 12:09:06.513058 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "f86445c074dc49e69f11336aba5db7d3"
format_stamp: "Formatted at 2026-08-09 12:09:06 on dist-test-slave-sb2z"
server_key: "8dbbf11074b89bf0e2912af3d0315f4f"
server_key_iv: "d8603fbc28b6607f9c6338a3f14b4215"
server_key_version: "encryptionkey@0"
I20260809 12:09:06.518045 31430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:09:06.521584 31456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:06.522924 31430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:09:06.523100 31430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "f86445c074dc49e69f11336aba5db7d3"
format_stamp: "Formatted at 2026-08-09 12:09:06 on dist-test-slave-sb2z"
server_key: "8dbbf11074b89bf0e2912af3d0315f4f"
server_key_iv: "d8603fbc28b6607f9c6338a3f14b4215"
server_key_version: "encryptionkey@0"
I20260809 12:09:06.523294 31430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:06.528934 31430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:06.533193 31430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:06.533488 31430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:06.552295 31430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36145
I20260809 12:09:06.552343 31541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36145 every 8 connection(s)
I20260809 12:09:06.553684 31430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:06.557585 31542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:06.558142 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 31430
I20260809 12:09:06.558327 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:06.558802 589 external_mini_cluster.cc:1428] Setting key a791db3a5e92b1dac8bb00d9fa1b7565
I20260809 12:09:06.568076 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:06.560792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:34647 (local address 127.0.0.1:36145)
0809 12:09:06.561625 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:06.561641 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:06.561697 (+ 56us) server_negotiation.cc:407] Connection header received
0809 12:09:06.564348 (+ 2651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:06.564376 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:06.564819 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:06.565256 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:06.566180 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:06.566188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:06.566205 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:06.566238 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:06.566298 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:06.566333 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:06.566551 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:06.566680 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 12:09:06.566871 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":333,"threads_started":1}
I20260809 12:09:06.571980 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:06.572537 31542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3: Bootstrap starting.
I20260809 12:09:06.576486 31542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:06.577833 31542 log.cc:824] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:06.580253 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:06.577388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:40977 (local address 127.0.0.1:36145)
0809 12:09:06.577612 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:06.577621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:06.578094 (+ 473us) server_negotiation.cc:407] Connection header received
0809 12:09:06.578267 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:06.578271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:06.578352 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:06.578494 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:06.579466 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:06.579474 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:06.579481 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:06.579502 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:06.579576 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:06.579602 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:06.579721 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:06.579860 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 12:09:06.579921 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:09:06.581321 31542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3: No bootstrap required, opened a new log
I20260809 12:09:06.588984 31542 raft_consensus.cc:348] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86445c074dc49e69f11336aba5db7d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260809 12:09:06.589485 31542 raft_consensus.cc:374] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:06.589614 31542 raft_consensus.cc:729] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f86445c074dc49e69f11336aba5db7d3, State: Initialized, Role: FOLLOWER
I20260809 12:09:06.590219 31542 consensus_queue.cc:260] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86445c074dc49e69f11336aba5db7d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260809 12:09:06.590476 31542 raft_consensus.cc:388] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:06.590624 31542 raft_consensus.cc:482] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:06.590801 31542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:06.593045 31542 raft_consensus.cc:504] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86445c074dc49e69f11336aba5db7d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260809 12:09:06.593531 31542 leader_election.cc:302] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f86445c074dc49e69f11336aba5db7d3; no voters:
I20260809 12:09:06.594082 31542 leader_election.cc:288] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:06.594275 31554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:06.594954 31554 raft_consensus.cc:686] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [term 1 LEADER]: Becoming Leader. State: Replica: f86445c074dc49e69f11336aba5db7d3, State: Running, Role: LEADER
I20260809 12:09:06.595582 31554 consensus_queue.cc:237] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86445c074dc49e69f11336aba5db7d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260809 12:09:06.596468 31542 sys_catalog.cc:564] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:06.598664 31556 sys_catalog.cc:455] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f86445c074dc49e69f11336aba5db7d3. Latest consensus state: current_term: 1 leader_uuid: "f86445c074dc49e69f11336aba5db7d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86445c074dc49e69f11336aba5db7d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } } }
I20260809 12:09:06.598639 31555 sys_catalog.cc:455] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f86445c074dc49e69f11336aba5db7d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86445c074dc49e69f11336aba5db7d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } } }
I20260809 12:09:06.599179 31556 sys_catalog.cc:458] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:06.599207 31555 sys_catalog.cc:458] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:06.601573 31564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:06.607734 31564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:06.616605 31564 catalog_manager.cc:1347] Generated new cluster ID: 77e72c8b370c4ad6bd2872878293b103
I20260809 12:09:06.616746 31564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:06.649556 31564 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:06.650054 31564 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:06.672845 31564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f86445c074dc49e69f11336aba5db7d3: Generated new TSK 0
I20260809 12:09:06.673733 31564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:06.689198 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31430
W20260809 12:09:06.690761 31533 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.157:34647
W20260809 12:09:06.691496 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31430: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:09:06.691684 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 31430
2026-08-09T12:09:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2236 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:06 dist-test-slave-sb2z krb5kdc[31584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:06 dist-test-slave-sb2z krb5kdc[31584](info): set up 2 sockets
Aug 09 12:09:06 dist-test-slave-sb2z krb5kdc[31584](info): commencing operation
krb5kdc: starting...
W20260809 12:09:08.032024 589 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.007s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:08 dist-test-slave-sb2z krb5kdc[31584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:09:08.375288 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34471
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:08.721294 31603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:08.721570 31603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:08.721663 31603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:08.728780 31603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:08.728879 31603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:08.728924 31603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:08.728977 31603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:08.729022 31603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:08.738495 31603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34471
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:08.739799 31603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:08.741017 31603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:08.752626 31623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:08.752727 31613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:08.752691 31615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:08.753716 31603 server_base.cc:1034] running on GCE node
I20260809 12:09:08.754824 31603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:08.756347 31603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:08.757596 31603 hybrid_clock.cc:648] HybridClock initialized: now 1786277348757555 us; error 53 us; skew 500 ppm
I20260809 12:09:08.757946 31603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:08 dist-test-slave-sb2z krb5kdc[31584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277348, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:08.761170 31603 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:09:08.762979 31603 webserver.cc:460] Webserver started at http://127.0.0.1:40359/ using document root <none> and password file <none>
I20260809 12:09:08.763708 31603 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:08.763828 31603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:08.764113 31603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:08.767051 31603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c"
format_stamp: "Formatted at 2026-08-09 12:09:08 on dist-test-slave-sb2z"
server_key: "108912d0017bdea833f8d446d3af1d6f"
server_key_iv: "cd7d8fa9fa4229a6ff929fa107405e4b"
server_key_version: "encryptionkey@0"
I20260809 12:09:08.767860 31603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c"
format_stamp: "Formatted at 2026-08-09 12:09:08 on dist-test-slave-sb2z"
server_key: "108912d0017bdea833f8d446d3af1d6f"
server_key_iv: "cd7d8fa9fa4229a6ff929fa107405e4b"
server_key_version: "encryptionkey@0"
I20260809 12:09:08.773025 31603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:09:08.776470 31635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:08.777822 31603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:09:08.777993 31603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c"
format_stamp: "Formatted at 2026-08-09 12:09:08 on dist-test-slave-sb2z"
server_key: "108912d0017bdea833f8d446d3af1d6f"
server_key_iv: "cd7d8fa9fa4229a6ff929fa107405e4b"
server_key_version: "encryptionkey@0"
I20260809 12:09:08.778159 31603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:08.805610 31603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:08.809576 31603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:08.809828 31603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:08.828513 31603 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34471
I20260809 12:09:08.828586 31709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34471 every 8 connection(s)
I20260809 12:09:08.829957 31603 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:08.833540 31710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:08.840021 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 31603
I20260809 12:09:08.840231 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:08.840685 589 external_mini_cluster.cc:1428] Setting key 3aa338fa2b51f48219d2fe6cf9853745
I20260809 12:09:08.844113 31710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c: Bootstrap starting.
I20260809 12:09:08.847558 31710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:08.848733 31710 log.cc:824] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:08.851975 31710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c: No bootstrap required, opened a new log
Aug 09 12:09:08 dist-test-slave-sb2z krb5kdc[31584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 12:09:08.861280 31710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } }
I20260809 12:09:08.861687 31710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:08.861811 31710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2331f37cc29a46e2a8a5f5ca03db7f4c, State: Initialized, Role: FOLLOWER
I20260809 12:09:08.862488 31710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } }
I20260809 12:09:08.862782 31710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:08.862946 31710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:08.863168 31710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:08.865680 31713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:08.842653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:35257 (local address 127.0.0.1:34471)
0809 12:09:08.843437 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:08.843448 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:08.843490 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:09:08.845335 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:08.845360 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:08.845742 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:08.846051 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:08.848055 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:08.849644 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:08.851449 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:08.852194 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:08.852334 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:08.857333 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:08.857341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:08.857355 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:08.857397 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:08.860625 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:08.861465 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:08.861471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:08.861477 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:08.861665 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:08.862250 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:08.862255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:08.862258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:08.862973 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:08.863312 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:08.863870 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 12:09:08.864290 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":367,"threads_started":1}
I20260809 12:09:08.866595 31710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } }
I20260809 12:09:08.867152 31710 leader_election.cc:302] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2331f37cc29a46e2a8a5f5ca03db7f4c; no voters:
I20260809 12:09:08.867822 31710 leader_election.cc:288] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:08.867951 31719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:08.869108 31719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [term 1 LEADER]: Becoming Leader. State: Replica: 2331f37cc29a46e2a8a5f5ca03db7f4c, State: Running, Role: LEADER
I20260809 12:09:08.869948 31719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } }
I20260809 12:09:08.870316 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:08.870376 31710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:08.872967 31720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } } }
I20260809 12:09:08.873442 31720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:08.876503 31722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2331f37cc29a46e2a8a5f5ca03db7f4c. Latest consensus state: current_term: 1 leader_uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2331f37cc29a46e2a8a5f5ca03db7f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34471 } } }
I20260809 12:09:08.876874 31722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:08.882532 31729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:08.888468 31729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:08.902058 31713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:08.880955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:43025 (local address 127.0.0.1:34471)
0809 12:09:08.881201 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:08.881209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:08.884124 (+ 2915us) server_negotiation.cc:407] Connection header received
0809 12:09:08.884383 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:08.884391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:08.884489 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:08.884573 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:08.886828 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:08.887830 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:08.889763 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:08.890472 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:08.890586 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:08.894073 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:08.894081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:08.894086 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:08.894124 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:08.898043 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:08.899109 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:08.899115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:08.899119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:08.899263 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:08.899934 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:08.899940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:08.899943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:08.900403 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:08.900684 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:08.901456 (+ 772us) server_negotiation.cc:299] Negotiation successful
0809 12:09:08.901660 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:09:08.906500 31729 catalog_manager.cc:1347] Generated new cluster ID: fe63420144d8411ab1cabbb2ffa6d64c
I20260809 12:09:08.906692 31729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:08.922175 31729 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:08.923496 31729 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:08.932026 31729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2331f37cc29a46e2a8a5f5ca03db7f4c: Generated new TSK 0
I20260809 12:09:08.933012 31729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:08.941746 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31603
I20260809 12:09:08.943751 31705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:09.034725 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 31603
W20260809 12:09:09.060086 31580 connection.cc:535] client connection to 127.0.0.1:34471 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34471): Connection reset by peer (error 104)
2026-08-09T12:09:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:09 dist-test-slave-sb2z krb5kdc[31756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:09 dist-test-slave-sb2z krb5kdc[31756](info): set up 2 sockets
Aug 09 12:09:09 dist-test-slave-sb2z krb5kdc[31756](info): commencing operation
krb5kdc: starting...
W20260809 12:09:10.411278 589 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:10 dist-test-slave-sb2z krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:09:10.755600 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:11.102303 31774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:11.102646 31774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:11.102802 31774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:11.110206 31774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:11.110361 31774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:11.110466 31774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:11.110579 31774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:11.110656 31774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:11.120443 31774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45905
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:11.121723 31774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:11.123018 31774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:11.134824 31782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:11.134920 31783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:11.134850 31786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:11.136198 31774 server_base.cc:1034] running on GCE node
I20260809 12:09:11.137010 31774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:11.138381 31774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:11.139617 31774 hybrid_clock.cc:648] HybridClock initialized: now 1786277351139576 us; error 59 us; skew 500 ppm
I20260809 12:09:11.139957 31774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:11 dist-test-slave-sb2z krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277351, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:11.143253 31774 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:09:11.144798 31774 webserver.cc:460] Webserver started at http://127.0.0.1:43787/ using document root <none> and password file <none>
I20260809 12:09:11.145486 31774 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:11.145630 31774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:11.145948 31774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:11.148931 31774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "703a8e18674f4cbd910e1ce613f9dadc"
format_stamp: "Formatted at 2026-08-09 12:09:11 on dist-test-slave-sb2z"
server_key: "1c097394a8b1adf658991833347baf9e"
server_key_iv: "04f4f0cab46ec6412d02a9dd3415f95b"
server_key_version: "encryptionkey@0"
I20260809 12:09:11.149711 31774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "703a8e18674f4cbd910e1ce613f9dadc"
format_stamp: "Formatted at 2026-08-09 12:09:11 on dist-test-slave-sb2z"
server_key: "1c097394a8b1adf658991833347baf9e"
server_key_iv: "04f4f0cab46ec6412d02a9dd3415f95b"
server_key_version: "encryptionkey@0"
I20260809 12:09:11.155040 31774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:09:11.159008 31799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:11.160440 31774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:09:11.160624 31774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "703a8e18674f4cbd910e1ce613f9dadc"
format_stamp: "Formatted at 2026-08-09 12:09:11 on dist-test-slave-sb2z"
server_key: "1c097394a8b1adf658991833347baf9e"
server_key_iv: "04f4f0cab46ec6412d02a9dd3415f95b"
server_key_version: "encryptionkey@0"
I20260809 12:09:11.160811 31774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:11.173197 31774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:11.177377 31774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:11.177662 31774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:11.196336 31774 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45905
I20260809 12:09:11.196380 31875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45905 every 8 connection(s)
I20260809 12:09:11.197701 31774 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:11.200330 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 31774
I20260809 12:09:11.200531 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:11.201119 589 external_mini_cluster.cc:1428] Setting key 362359be829b87dc72b332191e5185b4
I20260809 12:09:11.201905 31876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:11.215411 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:11.202899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:51757 (local address 127.0.0.1:45905)
0809 12:09:11.203846 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:11.203857 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:11.203895 (+ 38us) server_negotiation.cc:407] Connection header received
0809 12:09:11.206355 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:11.206377 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:11.206773 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:11.207102 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:11.207706 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:11.209010 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:11.210906 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:11.211647 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:11.211778 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:11.213079 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:11.213090 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:11.213103 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:11.213735 (+ 632us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:11.213817 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:11.213843 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:11.213980 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:11.214283 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 12:09:11.214487 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":409,"threads_started":1}
I20260809 12:09:11.215750 31876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc: Bootstrap starting.
I20260809 12:09:11.218959 31876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:11.219304 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:11.220276 31876 log.cc:824] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:11.223678 31876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc: No bootstrap required, opened a new log
I20260809 12:09:11.231280 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:11.224504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:58261 (local address 127.0.0.1:45905)
0809 12:09:11.224706 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:11.224712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:11.225045 (+ 333us) server_negotiation.cc:407] Connection header received
0809 12:09:11.225244 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:11.225247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:11.225307 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:11.225384 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:11.226011 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:11.226785 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:11.228677 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:11.229210 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:11.229315 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:11.230479 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:11.230484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:11.230487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:11.230503 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:11.230541 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:11.230559 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:11.230643 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:11.230863 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 12:09:11.230931 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:09:11.233379 31876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703a8e18674f4cbd910e1ce613f9dadc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45905 } }
I20260809 12:09:11.233753 31876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:11.233865 31876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 703a8e18674f4cbd910e1ce613f9dadc, State: Initialized, Role: FOLLOWER
I20260809 12:09:11.234457 31876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703a8e18674f4cbd910e1ce613f9dadc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45905 } }
I20260809 12:09:11.234684 31876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:11.234836 31876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:11.235014 31876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:11.237269 31876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703a8e18674f4cbd910e1ce613f9dadc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45905 } }
I20260809 12:09:11.237761 31876 leader_election.cc:302] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 703a8e18674f4cbd910e1ce613f9dadc; no voters:
I20260809 12:09:11.238310 31876 leader_election.cc:288] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:11.238449 31890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:11.240013 31890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [term 1 LEADER]: Becoming Leader. State: Replica: 703a8e18674f4cbd910e1ce613f9dadc, State: Running, Role: LEADER
I20260809 12:09:11.240605 31876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:11.240599 31890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703a8e18674f4cbd910e1ce613f9dadc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45905 } }
I20260809 12:09:11.243765 31894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 703a8e18674f4cbd910e1ce613f9dadc. Latest consensus state: current_term: 1 leader_uuid: "703a8e18674f4cbd910e1ce613f9dadc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703a8e18674f4cbd910e1ce613f9dadc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45905 } } }
I20260809 12:09:11.243738 31891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "703a8e18674f4cbd910e1ce613f9dadc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703a8e18674f4cbd910e1ce613f9dadc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45905 } } }
I20260809 12:09:11.244235 31894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:11.244241 31891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:11.245147 31900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:11.250736 31900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:11.259737 31900 catalog_manager.cc:1347] Generated new cluster ID: bedc6c0f07044327aa77c61cc96414b9
I20260809 12:09:11.259843 31900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:11.294826 31900 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:11.296823 31900 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:11.303242 31900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 703a8e18674f4cbd910e1ce613f9dadc: Generated new TSK 0
I20260809 12:09:11.304299 31900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:11.321369 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31774
W20260809 12:09:11.323122 31871 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.157:51757
W20260809 12:09:11.324092 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31774: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:09:11.324338 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 31774
W20260809 12:09:11.348242 31754 connection.cc:535] client connection to 127.0.0.1:45905 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45905): Connection reset by peer (error 104)
2026-08-09T12:09:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2287 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:11 dist-test-slave-sb2z krb5kdc[31920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:11 dist-test-slave-sb2z krb5kdc[31920](info): set up 2 sockets
Aug 09 12:09:11 dist-test-slave-sb2z krb5kdc[31920](info): commencing operation
krb5kdc: starting...
W20260809 12:09:12.640756 589 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.007s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:12 dist-test-slave-sb2z krb5kdc[31920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:09:12.965265 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43877
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:13.309355 31944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:13.309621 31944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:13.309794 31944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:13.317188 31944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:13.317339 31944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:13.317430 31944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:13.317530 31944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:13.317600 31944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:13.330776 31944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43877
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:13.332042 31944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:13.333315 31944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:13.345235 31951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:13.345153 31952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:13.345207 31955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:13.346146 31944 server_base.cc:1034] running on GCE node
I20260809 12:09:13.346853 31944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:13.348213 31944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:13.349442 31944 hybrid_clock.cc:648] HybridClock initialized: now 1786277353349410 us; error 44 us; skew 500 ppm
I20260809 12:09:13.349785 31944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:13 dist-test-slave-sb2z krb5kdc[31920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277353, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:13.357499 31944 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:09:13.359083 31944 webserver.cc:460] Webserver started at http://127.0.0.1:39261/ using document root <none> and password file <none>
I20260809 12:09:13.359850 31944 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:13.359992 31944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:13.360319 31944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:13.363361 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "55d8a95cc2a64a57a12abedcbd5478c6"
format_stamp: "Formatted at 2026-08-09 12:09:13 on dist-test-slave-sb2z"
server_key: "fbed3361d3e2f3e6f93bebc49d4872af"
server_key_iv: "6449d7ebf1d457806f449f9ff9ba07db"
server_key_version: "encryptionkey@0"
I20260809 12:09:13.364194 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "55d8a95cc2a64a57a12abedcbd5478c6"
format_stamp: "Formatted at 2026-08-09 12:09:13 on dist-test-slave-sb2z"
server_key: "fbed3361d3e2f3e6f93bebc49d4872af"
server_key_iv: "6449d7ebf1d457806f449f9ff9ba07db"
server_key_version: "encryptionkey@0"
I20260809 12:09:13.369237 31944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:09:13.372670 31963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:13.373917 31944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:09:13.374104 31944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "55d8a95cc2a64a57a12abedcbd5478c6"
format_stamp: "Formatted at 2026-08-09 12:09:13 on dist-test-slave-sb2z"
server_key: "fbed3361d3e2f3e6f93bebc49d4872af"
server_key_iv: "6449d7ebf1d457806f449f9ff9ba07db"
server_key_version: "encryptionkey@0"
I20260809 12:09:13.374289 31944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:13.380017 31944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:13.383929 31944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:13.384215 31944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:13.402554 31944 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43877
I20260809 12:09:13.402645 32046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43877 every 8 connection(s)
I20260809 12:09:13.403954 31944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:13.407465 32047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:13.407886 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 31944
I20260809 12:09:13.408066 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:13.408594 589 external_mini_cluster.cc:1428] Setting key d1c7194bf9c8d9ccd311c1eeb7625885
I20260809 12:09:13.417819 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:13.410385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:53315 (local address 127.0.0.1:43877)
0809 12:09:13.411258 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:13.411272 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:13.411332 (+ 60us) server_negotiation.cc:407] Connection header received
0809 12:09:13.414121 (+ 2789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:13.414149 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:13.414583 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:13.415031 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:13.415976 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:13.415984 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:13.416000 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:13.416025 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:13.416085 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:13.416112 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:13.416319 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:13.416453 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 12:09:13.416631 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":344,"threads_started":1}
I20260809 12:09:13.421365 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:13.421644 32047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6: Bootstrap starting.
I20260809 12:09:13.424633 32047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:13.425845 32047 log.cc:824] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:13.428265 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:13.425661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:52327 (local address 127.0.0.1:43877)
0809 12:09:13.425798 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:13.425802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:13.426360 (+ 558us) server_negotiation.cc:407] Connection header received
0809 12:09:13.426536 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:13.426542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:13.426596 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:13.426704 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:13.427652 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:13.427658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:13.427662 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:13.427679 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:13.427720 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:13.427739 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:13.427859 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:13.427929 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 12:09:13.427988 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 12:09:13.429098 32047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6: No bootstrap required, opened a new log
I20260809 12:09:13.436326 32047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d8a95cc2a64a57a12abedcbd5478c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43877 } }
I20260809 12:09:13.436694 32047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:13.436770 32047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d8a95cc2a64a57a12abedcbd5478c6, State: Initialized, Role: FOLLOWER
I20260809 12:09:13.437315 32047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d8a95cc2a64a57a12abedcbd5478c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43877 } }
I20260809 12:09:13.437522 32047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:13.437629 32047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:13.437752 32047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:13.440107 32047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d8a95cc2a64a57a12abedcbd5478c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43877 } }
I20260809 12:09:13.440528 32047 leader_election.cc:302] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55d8a95cc2a64a57a12abedcbd5478c6; no voters:
I20260809 12:09:13.441133 32047 leader_election.cc:288] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:13.441294 32059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:13.441959 32059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [term 1 LEADER]: Becoming Leader. State: Replica: 55d8a95cc2a64a57a12abedcbd5478c6, State: Running, Role: LEADER
I20260809 12:09:13.442477 32059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d8a95cc2a64a57a12abedcbd5478c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43877 } }
I20260809 12:09:13.443141 32047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:13.445286 32060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55d8a95cc2a64a57a12abedcbd5478c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d8a95cc2a64a57a12abedcbd5478c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43877 } } }
I20260809 12:09:13.445616 32060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:13.445421 32061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55d8a95cc2a64a57a12abedcbd5478c6. Latest consensus state: current_term: 1 leader_uuid: "55d8a95cc2a64a57a12abedcbd5478c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d8a95cc2a64a57a12abedcbd5478c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43877 } } }
I20260809 12:09:13.445904 32061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:13.446462 32068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:13.453955 32068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:13.461697 32068 catalog_manager.cc:1347] Generated new cluster ID: 4e6cac1de43245f9ae58f8847339abb5
I20260809 12:09:13.461799 32068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:13.473588 32068 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:13.473997 32068 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:13.482399 32068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55d8a95cc2a64a57a12abedcbd5478c6: Generated new TSK 0
I20260809 12:09:13.483171 32068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:13.489842 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31944
W20260809 12:09:13.491014 32040 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.157:53315
W20260809 12:09:13.491714 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 31944: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:09:13.491865 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 31944
2026-08-09T12:09:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2171 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:13 dist-test-slave-sb2z krb5kdc[32092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:13 dist-test-slave-sb2z krb5kdc[32092](info): set up 2 sockets
Aug 09 12:09:13 dist-test-slave-sb2z krb5kdc[32092](info): commencing operation
krb5kdc: starting...
W20260809 12:09:14.807380 589 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:15 dist-test-slave-sb2z krb5kdc[32092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:09:15.152663 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45665
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:36423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:45665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:15.502025 32112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:15.502367 32112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:15.502466 32112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:15.509856 32112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:15.509974 32112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:15.510028 32112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:15.510093 32112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:15.510139 32112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:15.520135 32112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:36423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:45665
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:45665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:15.521476 32112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:15.522812 32112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:15.534822 32118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:15.534834 32121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:15.535006 32119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:15.536782 32112 server_base.cc:1034] running on GCE node
I20260809 12:09:15.537673 32112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:15.539378 32112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:15.540601 32112 hybrid_clock.cc:648] HybridClock initialized: now 1786277355540558 us; error 62 us; skew 500 ppm
I20260809 12:09:15.541033 32112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:15 dist-test-slave-sb2z krb5kdc[32092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277355, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:15.544509 32112 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:09:15.546269 32112 webserver.cc:460] Webserver started at http://127.0.147.126:36199/ using document root <none> and password file <none>
I20260809 12:09:15.547055 32112 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:15.547210 32112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:15.547612 32112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:15.550853 32112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "e600949d8cf44e20a2d9d90833362be2"
format_stamp: "Formatted at 2026-08-09 12:09:15 on dist-test-slave-sb2z"
server_key: "0ea6e3ff457180bb2dd00abe92e7c3bf"
server_key_iv: "f4258754500ea98c408b2525003d75de"
server_key_version: "encryptionkey@0"
I20260809 12:09:15.551753 32112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "e600949d8cf44e20a2d9d90833362be2"
format_stamp: "Formatted at 2026-08-09 12:09:15 on dist-test-slave-sb2z"
server_key: "0ea6e3ff457180bb2dd00abe92e7c3bf"
server_key_iv: "f4258754500ea98c408b2525003d75de"
server_key_version: "encryptionkey@0"
I20260809 12:09:15.557329 32112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:09:15.561014 32131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:15.562377 32112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:09:15.562579 32112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "e600949d8cf44e20a2d9d90833362be2"
format_stamp: "Formatted at 2026-08-09 12:09:15 on dist-test-slave-sb2z"
server_key: "0ea6e3ff457180bb2dd00abe92e7c3bf"
server_key_iv: "f4258754500ea98c408b2525003d75de"
server_key_version: "encryptionkey@0"
I20260809 12:09:15.562805 32112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:15.578850 32112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:15.583051 32112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:15.583374 32112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:15.602684 32112 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:45665
I20260809 12:09:15.602712 32208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:45665 every 8 connection(s)
I20260809 12:09:15.604120 32112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:15.607091 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 32112
I20260809 12:09:15.607299 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:15.607820 589 external_mini_cluster.cc:1428] Setting key 248cc9d56f5baa9107fa2094b8cde995
I20260809 12:09:15.608031 32210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:15.622665 32210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2: Bootstrap starting.
Aug 09 12:09:15 dist-test-slave-sb2z krb5kdc[32092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:09:15.626286 32210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:15.627388 32210 log.cc:824] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:15.630738 32210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2: No bootstrap required, opened a new log
I20260809 12:09:15.631922 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:15.609596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60929 (local address 127.0.147.126:45665)
0809 12:09:15.610403 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:15.610415 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:15.610462 (+ 47us) server_negotiation.cc:407] Connection header received
0809 12:09:15.612602 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:15.612622 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:15.612987 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:15.613353 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:15.615211 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:15.616556 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:15.618438 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:15.619138 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:15.619282 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:15.624347 (+ 5065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:15.624355 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:15.624368 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:15.624403 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:15.627472 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:15.628492 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:15.628501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:15.628508 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:15.628738 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:15.629252 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:15.629255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:15.629257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:15.629761 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:15.629988 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:15.630524 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 12:09:15.630832 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":367,"threads_started":1}
I20260809 12:09:15.635716 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:15.640473 32210 raft_consensus.cc:348] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e600949d8cf44e20a2d9d90833362be2" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45665 } }
I20260809 12:09:15.640954 32210 raft_consensus.cc:374] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:15.641094 32210 raft_consensus.cc:729] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600949d8cf44e20a2d9d90833362be2, State: Initialized, Role: FOLLOWER
I20260809 12:09:15.641731 32210 consensus_queue.cc:260] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e600949d8cf44e20a2d9d90833362be2" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45665 } }
I20260809 12:09:15.642071 32210 raft_consensus.cc:388] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:15.642251 32210 raft_consensus.cc:482] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:15.642418 32210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:15.645215 32210 raft_consensus.cc:504] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e600949d8cf44e20a2d9d90833362be2" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45665 } }
I20260809 12:09:15.645694 32210 leader_election.cc:302] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e600949d8cf44e20a2d9d90833362be2; no voters:
I20260809 12:09:15.646358 32210 leader_election.cc:288] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:15.646484 32228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:15.647601 32228 raft_consensus.cc:686] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [term 1 LEADER]: Becoming Leader. State: Replica: e600949d8cf44e20a2d9d90833362be2, State: Running, Role: LEADER
I20260809 12:09:15.648181 32228 consensus_queue.cc:237] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e600949d8cf44e20a2d9d90833362be2" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45665 } }
I20260809 12:09:15.648737 32210 sys_catalog.cc:564] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:15.652155 32232 sys_catalog.cc:455] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e600949d8cf44e20a2d9d90833362be2. Latest consensus state: current_term: 1 leader_uuid: "e600949d8cf44e20a2d9d90833362be2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e600949d8cf44e20a2d9d90833362be2" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45665 } } }
I20260809 12:09:15.653047 32232 sys_catalog.cc:458] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:15.652711 32231 sys_catalog.cc:455] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e600949d8cf44e20a2d9d90833362be2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e600949d8cf44e20a2d9d90833362be2" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 45665 } } }
I20260809 12:09:15.653656 32231 sys_catalog.cc:458] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:15.656446 32237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:15.659631 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:15.640996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48525 (local address 127.0.147.126:45665)
0809 12:09:15.641190 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:15.641196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:15.641651 (+ 455us) server_negotiation.cc:407] Connection header received
0809 12:09:15.641772 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:15.641778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:15.641851 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:15.641961 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:15.643586 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:15.644699 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:15.645932 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:15.646428 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:15.646574 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:15.649400 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:15.649405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:15.649408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:15.649437 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:15.654604 (+ 5167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:15.656013 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:15.656025 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:15.656032 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:15.656204 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:15.656962 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:15.656969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:15.656975 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:15.657553 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:15.657941 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:15.659032 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0809 12:09:15.659226 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:09:15.661926 32237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:15.678442 32237 catalog_manager.cc:1347] Generated new cluster ID: 6a6ecb8e9b3b46cd846fc39959b2ccf1
I20260809 12:09:15.678581 32237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:15.688529 32237 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:15.689734 32237 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:15.695621 32237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e600949d8cf44e20a2d9d90833362be2: Generated new TSK 0
I20260809 12:09:15.696542 32237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:15.707326 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 32112
I20260809 12:09:15.708541 32200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:15.797868 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 32112
W20260809 12:09:15.817909 32087 connection.cc:535] client connection to 127.0.147.126:45665 recv error: Network error: failed to read from TLS socket (remote: 127.0.147.126:45665): Connection reset by peer (error 104)
2026-08-09T12:09:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:15 dist-test-slave-sb2z krb5kdc[32257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:15 dist-test-slave-sb2z krb5kdc[32257](info): set up 2 sockets
Aug 09 12:09:15 dist-test-slave-sb2z krb5kdc[32257](info): commencing operation
krb5kdc: starting...
W20260809 12:09:17.111959 589 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:17 dist-test-slave-sb2z krb5kdc[32257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:09:17.445329 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:33197
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:45683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:33197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:17.806226 32277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:17.806527 32277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:17.806624 32277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:17.813803 32277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:17.813910 32277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:17.813957 32277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:17.814028 32277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:17.814071 32277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:17.823800 32277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:45683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:33197
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:33197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:17.825006 32277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:17.826220 32277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:17.838090 32284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:17.838172 32285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:17.838140 32288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:17.839074 32277 server_base.cc:1034] running on GCE node
I20260809 12:09:17.839916 32277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:17.841243 32277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:17.842481 32277 hybrid_clock.cc:648] HybridClock initialized: now 1786277357842437 us; error 47 us; skew 500 ppm
I20260809 12:09:17.842828 32277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:17 dist-test-slave-sb2z krb5kdc[32257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277357, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:17.846134 32277 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:09:17.847698 32277 webserver.cc:460] Webserver started at http://127.0.147.126:35671/ using document root <none> and password file <none>
I20260809 12:09:17.848367 32277 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:17.848520 32277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:17.848824 32277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:17.851852 32277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "c7cb691cb72547d292efa35059d74a4d"
format_stamp: "Formatted at 2026-08-09 12:09:17 on dist-test-slave-sb2z"
server_key: "3411a8eaf071a09efd8c4da5b3d123e4"
server_key_iv: "15cdcfc02a9cba8dff8b7b9f22561155"
server_key_version: "encryptionkey@0"
I20260809 12:09:17.852626 32277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "c7cb691cb72547d292efa35059d74a4d"
format_stamp: "Formatted at 2026-08-09 12:09:17 on dist-test-slave-sb2z"
server_key: "3411a8eaf071a09efd8c4da5b3d123e4"
server_key_iv: "15cdcfc02a9cba8dff8b7b9f22561155"
server_key_version: "encryptionkey@0"
I20260809 12:09:17.857738 32277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:09:17.861146 32296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:17.862425 32277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:09:17.862618 32277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "c7cb691cb72547d292efa35059d74a4d"
format_stamp: "Formatted at 2026-08-09 12:09:17 on dist-test-slave-sb2z"
server_key: "3411a8eaf071a09efd8c4da5b3d123e4"
server_key_iv: "15cdcfc02a9cba8dff8b7b9f22561155"
server_key_version: "encryptionkey@0"
I20260809 12:09:17.862828 32277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:17.887077 32277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:17.891251 32277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:17.891604 32277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:17.912025 32277 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:33197
I20260809 12:09:17.912050 32370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:33197 every 8 connection(s)
I20260809 12:09:17.913527 32277 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:17.917214 32372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:17.917753 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 32277
I20260809 12:09:17.917949 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:17.918571 589 external_mini_cluster.cc:1428] Setting key 1e3b82c0da5b8ab4d7a6678f99fb09ce
I20260809 12:09:17.929682 32372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d: Bootstrap starting.
I20260809 12:09:17.932009 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:17.920538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59385 (local address 127.0.147.126:33197)
0809 12:09:17.921354 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:17.921369 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:17.921411 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:09:17.923618 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:17.923640 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:17.924019 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:17.924403 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:17.924944 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:17.926239 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:17.927606 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:17.928261 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:17.928450 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:17.929774 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:17.929786 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:17.929807 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:17.930338 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:17.930412 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:17.930436 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:17.930612 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:17.930909 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 12:09:17.931078 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":345,"threads_started":1}
I20260809 12:09:17.932907 32372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:17.934273 32372 log.cc:824] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:17.936002 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:17.937367 32372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d: No bootstrap required, opened a new log
I20260809 12:09:17.947167 32372 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cb691cb72547d292efa35059d74a4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33197 } }
I20260809 12:09:17.947662 32372 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:17.947788 32372 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7cb691cb72547d292efa35059d74a4d, State: Initialized, Role: FOLLOWER
I20260809 12:09:17.948415 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:17.941377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59109 (local address 127.0.147.126:33197)
0809 12:09:17.941548 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:17.941554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:17.942040 (+ 486us) server_negotiation.cc:407] Connection header received
0809 12:09:17.942186 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:17.942191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:17.942270 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:17.942386 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:17.942909 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:17.944118 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:17.945550 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:17.946250 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:17.946380 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:17.947511 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:17.947518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:17.947552 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:17.947577 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:17.947623 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:17.947654 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:17.947794 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:17.948061 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 12:09:17.948117 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:09:17.948417 32372 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cb691cb72547d292efa35059d74a4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33197 } }
I20260809 12:09:17.948837 32372 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:17.949030 32372 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:17.949242 32372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:17.951855 32372 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cb691cb72547d292efa35059d74a4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33197 } }
I20260809 12:09:17.952370 32372 leader_election.cc:302] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7cb691cb72547d292efa35059d74a4d; no voters:
I20260809 12:09:17.952978 32372 leader_election.cc:288] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:17.953065 32385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:17.954288 32385 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [term 1 LEADER]: Becoming Leader. State: Replica: c7cb691cb72547d292efa35059d74a4d, State: Running, Role: LEADER
I20260809 12:09:17.954811 32385 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cb691cb72547d292efa35059d74a4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33197 } }
I20260809 12:09:17.954998 32372 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:17.957666 32387 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7cb691cb72547d292efa35059d74a4d. Latest consensus state: current_term: 1 leader_uuid: "c7cb691cb72547d292efa35059d74a4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cb691cb72547d292efa35059d74a4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33197 } } }
I20260809 12:09:17.957679 32386 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7cb691cb72547d292efa35059d74a4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cb691cb72547d292efa35059d74a4d" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 33197 } } }
I20260809 12:09:17.958107 32386 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:17.958107 32387 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:17.959059 32391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:17.964148 32391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:17.978116 32391 catalog_manager.cc:1347] Generated new cluster ID: 8b89635b936c494d846ad8820d1dbb11
I20260809 12:09:17.978238 32391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:17.996682 32391 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:17.997988 32391 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:18.010783 32391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7cb691cb72547d292efa35059d74a4d: Generated new TSK 0
I20260809 12:09:18.011744 32391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:18.028365 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 32277
W20260809 12:09:18.029810 32362 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59385
W20260809 12:09:18.030638 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 32277: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:09:18.030803 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 32277
W20260809 12:09:18.055608 32255 connection.cc:535] client connection to 127.0.147.126:33197 recv error: Network error: failed to read from TLS socket (remote: 127.0.147.126:33197): Connection reset by peer (error 104)
2026-08-09T12:09:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2237 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:18 dist-test-slave-sb2z krb5kdc[32409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:18 dist-test-slave-sb2z krb5kdc[32409](info): set up 2 sockets
Aug 09 12:09:18 dist-test-slave-sb2z krb5kdc[32409](info): commencing operation
krb5kdc: starting...
W20260809 12:09:19.342696 589 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.005s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:19 dist-test-slave-sb2z krb5kdc[32409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:09:19.696200 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39191
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:43697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:39191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:20.040753 32425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:20.041167 32425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:20.041363 32425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:20.049590 32425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:20.049742 32425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:20.049852 32425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:20.049975 32425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:20.050056 32425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:20.060751 32425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:43697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:39191
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:39191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:20.061997 32425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:20.063239 32425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:20.075398 32436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:20.075927 32439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:20.075497 32435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:20.076165 32425 server_base.cc:1034] running on GCE node
I20260809 12:09:20.077098 32425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:20.078471 32425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:20.079687 32425 hybrid_clock.cc:648] HybridClock initialized: now 1786277360079635 us; error 72 us; skew 500 ppm
I20260809 12:09:20.080042 32425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:20 dist-test-slave-sb2z krb5kdc[32409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277360, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:20.086376 32425 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:09:20.088003 32425 webserver.cc:460] Webserver started at http://127.0.147.126:36087/ using document root <none> and password file <none>
I20260809 12:09:20.088698 32425 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:20.088804 32425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:20.089062 32425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:20.092113 32425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "48e8423e143f406199d1869016f8a3be"
format_stamp: "Formatted at 2026-08-09 12:09:20 on dist-test-slave-sb2z"
server_key: "c8d12b52494adff631a9b325d1ac90d9"
server_key_iv: "e19571c2c482abdb3a99a9e59d222603"
server_key_version: "encryptionkey@0"
I20260809 12:09:20.092850 32425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "48e8423e143f406199d1869016f8a3be"
format_stamp: "Formatted at 2026-08-09 12:09:20 on dist-test-slave-sb2z"
server_key: "c8d12b52494adff631a9b325d1ac90d9"
server_key_iv: "e19571c2c482abdb3a99a9e59d222603"
server_key_version: "encryptionkey@0"
I20260809 12:09:20.097908 32425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:09:20.101269 32448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:20.102535 32425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:09:20.102684 32425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "48e8423e143f406199d1869016f8a3be"
format_stamp: "Formatted at 2026-08-09 12:09:20 on dist-test-slave-sb2z"
server_key: "c8d12b52494adff631a9b325d1ac90d9"
server_key_iv: "e19571c2c482abdb3a99a9e59d222603"
server_key_version: "encryptionkey@0"
I20260809 12:09:20.102845 32425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:20.108585 32425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:20.112425 32425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:20.112682 32425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:20.131202 32425 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:39191
I20260809 12:09:20.131258 32511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:39191 every 8 connection(s)
I20260809 12:09:20.132580 32425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:20.136193 32515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:20.137832 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 32425
I20260809 12:09:20.138028 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:20.138607 589 external_mini_cluster.cc:1428] Setting key e2fb01786360f5dc1b83990ffb86baf3
I20260809 12:09:20.146273 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:20.140446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58887 (local address 127.0.147.126:39191)
0809 12:09:20.141287 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:20.141298 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:20.141343 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:09:20.142962 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:20.142987 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:20.143328 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:20.143711 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:20.144769 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:20.144775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:20.144788 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:20.144802 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:20.144865 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:20.144883 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:20.145052 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:20.145174 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 12:09:20.145355 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":404,"threads_started":1}
I20260809 12:09:20.149961 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:20.149969 32515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be: Bootstrap starting.
I20260809 12:09:20.153813 32515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:20.155190 32515 log.cc:824] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:20.157874 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:20.155242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41255 (local address 127.0.147.126:39191)
0809 12:09:20.155412 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:20.155420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:20.156036 (+ 616us) server_negotiation.cc:407] Connection header received
0809 12:09:20.156181 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:20.156189 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:20.156254 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:20.156384 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:20.157243 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:20.157248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:20.157251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:20.157265 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:20.157298 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:20.157312 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:20.157396 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:20.157530 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 12:09:20.157587 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 12:09:20.158586 32515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be: No bootstrap required, opened a new log
I20260809 12:09:20.165815 32515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e8423e143f406199d1869016f8a3be" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39191 } }
I20260809 12:09:20.166181 32515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:20.166286 32515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e8423e143f406199d1869016f8a3be, State: Initialized, Role: FOLLOWER
I20260809 12:09:20.166832 32515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e8423e143f406199d1869016f8a3be" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39191 } }
I20260809 12:09:20.167088 32515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:20.167223 32515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:20.167395 32515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:20.169576 32515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e8423e143f406199d1869016f8a3be" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39191 } }
I20260809 12:09:20.170071 32515 leader_election.cc:302] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48e8423e143f406199d1869016f8a3be; no voters:
I20260809 12:09:20.170629 32515 leader_election.cc:288] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:20.170770 32528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:20.171459 32528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [term 1 LEADER]: Becoming Leader. State: Replica: 48e8423e143f406199d1869016f8a3be, State: Running, Role: LEADER
I20260809 12:09:20.172036 32528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e8423e143f406199d1869016f8a3be" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39191 } }
I20260809 12:09:20.172752 32515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:20.174975 32530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48e8423e143f406199d1869016f8a3be. Latest consensus state: current_term: 1 leader_uuid: "48e8423e143f406199d1869016f8a3be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e8423e143f406199d1869016f8a3be" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39191 } } }
I20260809 12:09:20.175592 32530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:20.175639 32529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48e8423e143f406199d1869016f8a3be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e8423e143f406199d1869016f8a3be" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 39191 } } }
I20260809 12:09:20.175978 32529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:20.176661 32542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:20.182689 32542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:20.191958 32542 catalog_manager.cc:1347] Generated new cluster ID: 81c10d3b62ee48c1b038b6917f67598b
I20260809 12:09:20.192068 32542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:20.217275 32542 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:20.217778 32542 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:20.229502 32542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48e8423e143f406199d1869016f8a3be: Generated new TSK 0
I20260809 12:09:20.230589 32542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:20.247247 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 32425
W20260809 12:09:20.248649 32507 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58887
W20260809 12:09:20.249362 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 32425: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:09:20.249521 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 32425
2026-08-09T12:09:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2220 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:20 dist-test-slave-sb2z krb5kdc[32564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:20 dist-test-slave-sb2z krb5kdc[32564](info): set up 2 sockets
Aug 09 12:09:20 dist-test-slave-sb2z krb5kdc[32564](info): commencing operation
krb5kdc: starting...
W20260809 12:09:21.613334 589 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:21 dist-test-slave-sb2z krb5kdc[32564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:09:21.966531 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:36871
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:36871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:22.319011 32583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:22.319296 32583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:22.319511 32583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:22.327317 32583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:22.327425 32583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:22.327576 32583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:22.327690 32583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:22.327778 32583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:22.337393 32583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:36871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:36871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:22.338709 32583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:22.340019 32583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:22.352056 32590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:22.351984 32592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:22.352077 32594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:22.352757 32583 server_base.cc:1034] running on GCE node
I20260809 12:09:22.353585 32583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:22.354998 32583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:22.356233 32583 hybrid_clock.cc:648] HybridClock initialized: now 1786277362356175 us; error 43 us; skew 500 ppm
I20260809 12:09:22.356580 32583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:22 dist-test-slave-sb2z krb5kdc[32564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277362, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:22.359879 32583 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:09:22.361337 32583 webserver.cc:460] Webserver started at http://127.0.147.126:41615/ using document root <none> and password file <none>
I20260809 12:09:22.362023 32583 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:22.362157 32583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:22.362463 32583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:22.365437 32583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc"
format_stamp: "Formatted at 2026-08-09 12:09:22 on dist-test-slave-sb2z"
server_key: "decf9d1315c7002865d67963212d588b"
server_key_iv: "af37bb131102bb59781a19e207144169"
server_key_version: "encryptionkey@0"
I20260809 12:09:22.366225 32583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc"
format_stamp: "Formatted at 2026-08-09 12:09:22 on dist-test-slave-sb2z"
server_key: "decf9d1315c7002865d67963212d588b"
server_key_iv: "af37bb131102bb59781a19e207144169"
server_key_version: "encryptionkey@0"
I20260809 12:09:22.371240 32583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:09:22.374704 32605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:22.375985 32583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 12:09:22.376173 32583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc"
format_stamp: "Formatted at 2026-08-09 12:09:22 on dist-test-slave-sb2z"
server_key: "decf9d1315c7002865d67963212d588b"
server_key_iv: "af37bb131102bb59781a19e207144169"
server_key_version: "encryptionkey@0"
I20260809 12:09:22.376369 32583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:22.398135 32583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:22.402138 32583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:22.402442 32583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:22.421375 32583 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:36871
I20260809 12:09:22.421411 32680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:36871 every 8 connection(s)
I20260809 12:09:22.422746 32583 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:22.426383 32681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:22.430651 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 32583
I20260809 12:09:22.430840 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:22.431274 589 external_mini_cluster.cc:1428] Setting key f4e5b7393fed2a024ffc53490b0772a1
I20260809 12:09:22.437251 32681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc: Bootstrap starting.
I20260809 12:09:22.440873 32681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:22.442099 32681 log.cc:824] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:22.444880 32681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc: No bootstrap required, opened a new log
Aug 09 12:09:22 dist-test-slave-sb2z krb5kdc[32564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:09:22.454779 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:22.433059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60601 (local address 127.0.147.126:36871)
0809 12:09:22.433850 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:22.433860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:22.433902 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:09:22.435753 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:22.435775 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:22.436128 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:22.436429 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:22.438330 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:22.439754 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:22.441619 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:22.442328 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:22.442460 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:22.447248 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:22.447256 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:22.447269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:22.447312 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:22.450550 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:22.451386 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:22.451392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:22.451398 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:22.451629 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:22.452054 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:22.452057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:22.452059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:22.452555 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:22.452790 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:22.453276 (+ 486us) server_negotiation.cc:299] Negotiation successful
0809 12:09:22.453646 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":387,"threads_started":1}
I20260809 12:09:22.454597 32681 raft_consensus.cc:348] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36871 } }
I20260809 12:09:22.454994 32681 raft_consensus.cc:374] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:22.455185 32681 raft_consensus.cc:729] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edc1a2d2ffa7406c8220d248bfdd1bcc, State: Initialized, Role: FOLLOWER
I20260809 12:09:22.455816 32681 consensus_queue.cc:260] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36871 } }
I20260809 12:09:22.456082 32681 raft_consensus.cc:388] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:22.456259 32681 raft_consensus.cc:482] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:22.456471 32681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:22.459033 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:22.459467 32681 raft_consensus.cc:504] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36871 } }
I20260809 12:09:22.460021 32681 leader_election.cc:302] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edc1a2d2ffa7406c8220d248bfdd1bcc; no voters:
I20260809 12:09:22.460634 32681 leader_election.cc:288] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:22.460772 32691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:22.462860 32681 sys_catalog.cc:564] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:22.464133 32691 raft_consensus.cc:686] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [term 1 LEADER]: Becoming Leader. State: Replica: edc1a2d2ffa7406c8220d248bfdd1bcc, State: Running, Role: LEADER
I20260809 12:09:22.464974 32691 consensus_queue.cc:237] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36871 } }
I20260809 12:09:22.468884 32693 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36871 } } }
I20260809 12:09:22.469375 32693 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:22.470041 32695 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader edc1a2d2ffa7406c8220d248bfdd1bcc. Latest consensus state: current_term: 1 leader_uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1a2d2ffa7406c8220d248bfdd1bcc" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 36871 } } }
I20260809 12:09:22.470398 32695 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:22.473503 32708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:22.478834 32708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:22.494545 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:22.470249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56051 (local address 127.0.147.126:36871)
0809 12:09:22.471202 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:22.471211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:22.471904 (+ 693us) server_negotiation.cc:407] Connection header received
0809 12:09:22.472128 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:22.472136 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:22.472240 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:22.472364 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:22.474667 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:22.476293 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:22.478778 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:22.479798 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:22.479960 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:22.484191 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:22.484201 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:22.484209 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:22.484263 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:22.489321 (+ 5058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:22.490461 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:22.490474 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:22.490481 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:22.490702 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:22.491298 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:22.491306 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:22.491311 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:22.491849 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:22.492169 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:22.493658 (+ 1489us) server_negotiation.cc:299] Negotiation successful
0809 12:09:22.493964 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790}
I20260809 12:09:22.496338 32708 catalog_manager.cc:1347] Generated new cluster ID: 1a750d24f3b041dda1e12ccb85ba10b3
I20260809 12:09:22.496507 32708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:22.516448 32708 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:22.517979 32708 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:22.523782 32708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edc1a2d2ffa7406c8220d248bfdd1bcc: Generated new TSK 0
I20260809 12:09:22.524564 32708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:22.529999 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 32583
I20260809 12:09:22.531148 32673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:22.620102 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 32583
2026-08-09T12:09:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:22 dist-test-slave-sb2z krb5kdc[32732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:22 dist-test-slave-sb2z krb5kdc[32732](info): set up 2 sockets
Aug 09 12:09:22 dist-test-slave-sb2z krb5kdc[32732](info): commencing operation
krb5kdc: starting...
W20260809 12:09:23.942818 589 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:24 dist-test-slave-sb2z krb5kdc[32732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:09:24.292631 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:44781
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:44781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:24.644650 32748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:24.644992 32748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:24.645157 32748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:24.652388 32748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:24.652542 32748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:24.652648 32748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:24.652756 32748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:24.652833 32748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:24.662797 32748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:44781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:44781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:24.664211 32748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:24.665485 32748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:24.677753 32754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:24.677778 32755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:24.677872 32757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:24.678536 32748 server_base.cc:1034] running on GCE node
I20260809 12:09:24.679381 32748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:24.680876 32748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:24.682197 32748 hybrid_clock.cc:648] HybridClock initialized: now 1786277364682135 us; error 52 us; skew 500 ppm
I20260809 12:09:24.682559 32748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:24 dist-test-slave-sb2z krb5kdc[32732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277364, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:24.685803 32748 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:09:24.687150 32748 webserver.cc:460] Webserver started at http://127.0.147.126:46465/ using document root <none> and password file <none>
I20260809 12:09:24.687865 32748 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:24.687970 32748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:24.688213 32748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:24.691069 32748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "db18673cbe9d47c7aede6acc1953061f"
format_stamp: "Formatted at 2026-08-09 12:09:24 on dist-test-slave-sb2z"
server_key: "ed0b396c47eeec6de133c6d097b2bf37"
server_key_iv: "ffed5ee0543a6459031f3b8902714048"
server_key_version: "encryptionkey@0"
I20260809 12:09:24.691876 32748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "db18673cbe9d47c7aede6acc1953061f"
format_stamp: "Formatted at 2026-08-09 12:09:24 on dist-test-slave-sb2z"
server_key: "ed0b396c47eeec6de133c6d097b2bf37"
server_key_iv: "ffed5ee0543a6459031f3b8902714048"
server_key_version: "encryptionkey@0"
I20260809 12:09:24.696957 32748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:09:24.700274 300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:24.701545 32748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:09:24.701726 32748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "db18673cbe9d47c7aede6acc1953061f"
format_stamp: "Formatted at 2026-08-09 12:09:24 on dist-test-slave-sb2z"
server_key: "ed0b396c47eeec6de133c6d097b2bf37"
server_key_iv: "ffed5ee0543a6459031f3b8902714048"
server_key_version: "encryptionkey@0"
I20260809 12:09:24.701920 32748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:24.720728 32748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:24.724834 32748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:24.725116 32748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:24.744012 32748 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:44781
I20260809 12:09:24.744045 370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:44781 every 8 connection(s)
I20260809 12:09:24.745379 32748 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:24.749010 371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:24.755234 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 32748
I20260809 12:09:24.755456 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:24.756137 589 external_mini_cluster.cc:1428] Setting key c72113466dc4c647cb19ecfabd98951d
I20260809 12:09:24.760229 371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f: Bootstrap starting.
I20260809 12:09:24.763701 371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:24.764935 371 log.cc:824] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:24.768214 371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f: No bootstrap required, opened a new log
I20260809 12:09:24.770013 374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:24.757965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49809 (local address 127.0.147.126:44781)
0809 12:09:24.758802 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:24.758837 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:24.758892 (+ 55us) server_negotiation.cc:407] Connection header received
0809 12:09:24.760688 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:24.760714 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:24.761115 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:24.761415 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:24.762052 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:24.763319 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:24.765253 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:24.766383 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:24.766509 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:24.767776 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:24.767783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:24.767797 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:24.768351 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:24.768441 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:24.768468 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:24.768564 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:24.768800 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 12:09:24.769050 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":386,"threads_started":1}
I20260809 12:09:24.773680 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:24.777451 371 raft_consensus.cc:348] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db18673cbe9d47c7aede6acc1953061f" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44781 } }
I20260809 12:09:24.777964 371 raft_consensus.cc:374] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:24.778088 371 raft_consensus.cc:729] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db18673cbe9d47c7aede6acc1953061f, State: Initialized, Role: FOLLOWER
I20260809 12:09:24.778760 371 consensus_queue.cc:260] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db18673cbe9d47c7aede6acc1953061f" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44781 } }
I20260809 12:09:24.779076 371 raft_consensus.cc:388] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:24.779295 371 raft_consensus.cc:482] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:24.779513 371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:24.782778 371 raft_consensus.cc:504] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db18673cbe9d47c7aede6acc1953061f" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44781 } }
I20260809 12:09:24.783366 371 leader_election.cc:302] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db18673cbe9d47c7aede6acc1953061f; no voters:
I20260809 12:09:24.784078 371 leader_election.cc:288] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:24.784252 385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:24.785041 385 raft_consensus.cc:686] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [term 1 LEADER]: Becoming Leader. State: Replica: db18673cbe9d47c7aede6acc1953061f, State: Running, Role: LEADER
I20260809 12:09:24.786103 385 consensus_queue.cc:237] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db18673cbe9d47c7aede6acc1953061f" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44781 } }
I20260809 12:09:24.786964 374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:24.779092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39901 (local address 127.0.147.126:44781)
0809 12:09:24.779280 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:24.779289 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:24.779902 (+ 613us) server_negotiation.cc:407] Connection header received
0809 12:09:24.780096 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:24.780099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:24.780162 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:24.780241 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:24.780836 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:24.781603 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:24.783445 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:24.784255 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:24.784396 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:24.785759 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:24.785766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:24.785770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:24.785790 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:24.785835 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:24.785853 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:24.785953 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:24.786543 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 12:09:24.786639 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:09:24.787214 371 sys_catalog.cc:564] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:24.789978 386 sys_catalog.cc:455] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [sys.catalog]: SysCatalogTable state changed. Reason: New leader db18673cbe9d47c7aede6acc1953061f. Latest consensus state: current_term: 1 leader_uuid: "db18673cbe9d47c7aede6acc1953061f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db18673cbe9d47c7aede6acc1953061f" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44781 } } }
I20260809 12:09:24.789950 387 sys_catalog.cc:455] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db18673cbe9d47c7aede6acc1953061f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db18673cbe9d47c7aede6acc1953061f" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44781 } } }
I20260809 12:09:24.790407 386 sys_catalog.cc:458] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:24.790477 387 sys_catalog.cc:458] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:24.791745 393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:24.796926 393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:24.805398 393 catalog_manager.cc:1347] Generated new cluster ID: 318584f8069c402db4cc1f1d46774b41
I20260809 12:09:24.805529 393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:24.813397 393 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:24.814550 393 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:24.826711 393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db18673cbe9d47c7aede6acc1953061f: Generated new TSK 0
I20260809 12:09:24.827509 393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:24.840268 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 32748
W20260809 12:09:24.841742 365 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49809
W20260809 12:09:24.842459 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 32748: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:09:24.842629 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 32748
2026-08-09T12:09:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2226 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:24 dist-test-slave-sb2z krb5kdc[413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:24 dist-test-slave-sb2z krb5kdc[413](info): set up 2 sockets
Aug 09 12:09:24 dist-test-slave-sb2z krb5kdc[413](info): commencing operation
krb5kdc: starting...
W20260809 12:09:26.167927 589 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.004s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:26 dist-test-slave-sb2z krb5kdc[413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:09:26.516939 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:46639
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:46639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:26.877799 432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:26.878090 432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:26.878182 432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:26.885306 432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:26.885413 432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:26.885463 432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:26.885526 432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:26.885571 432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:26.895156 432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:46639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:46639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:26.896447 432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:26.897712 432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:26.909745 443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:26.909937 439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:26.909911 440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:26.910971 432 server_base.cc:1034] running on GCE node
I20260809 12:09:26.911856 432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:26.913153 432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:26.914382 432 hybrid_clock.cc:648] HybridClock initialized: now 1786277366914337 us; error 74 us; skew 500 ppm
I20260809 12:09:26.914733 432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:26 dist-test-slave-sb2z krb5kdc[413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:26.921912 432 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:09:26.923306 432 webserver.cc:460] Webserver started at http://127.0.147.126:37123/ using document root <none> and password file <none>
I20260809 12:09:26.924022 432 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:26.924126 432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:26.924376 432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:26.927255 432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "2beb98234e6a4a01843a0679596892d7"
format_stamp: "Formatted at 2026-08-09 12:09:26 on dist-test-slave-sb2z"
server_key: "474b9eafe5b1a33cfae4ea2a639d5700"
server_key_iv: "9aa82a9c02af9f244f5575430bc5227b"
server_key_version: "encryptionkey@0"
I20260809 12:09:26.928047 432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "2beb98234e6a4a01843a0679596892d7"
format_stamp: "Formatted at 2026-08-09 12:09:26 on dist-test-slave-sb2z"
server_key: "474b9eafe5b1a33cfae4ea2a639d5700"
server_key_iv: "9aa82a9c02af9f244f5575430bc5227b"
server_key_version: "encryptionkey@0"
I20260809 12:09:26.933162 432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:09:26.936491 452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:26.937794 432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:09:26.937958 432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "2beb98234e6a4a01843a0679596892d7"
format_stamp: "Formatted at 2026-08-09 12:09:26 on dist-test-slave-sb2z"
server_key: "474b9eafe5b1a33cfae4ea2a639d5700"
server_key_iv: "9aa82a9c02af9f244f5575430bc5227b"
server_key_version: "encryptionkey@0"
I20260809 12:09:26.938163 432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:26.944360 432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:26.948643 432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:26.948931 432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:26.968073 432 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:46639
I20260809 12:09:26.968120 529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:46639 every 8 connection(s)
I20260809 12:09:26.969508 432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:26.970378 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 432
I20260809 12:09:26.970602 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:26.971033 589 external_mini_cluster.cc:1428] Setting key 6d61b485cf9b8916d0cec00049b77d2a
I20260809 12:09:26.973861 530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:26.978819 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:26.972855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39155 (local address 127.0.147.126:46639)
0809 12:09:26.973751 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:26.973763 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:26.973804 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:09:26.975474 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:26.975495 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:26.975882 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:26.976197 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:26.977287 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:26.977293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:26.977304 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:26.977319 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:26.977388 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:26.977410 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:26.977590 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:26.977718 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 12:09:26.977911 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":388,"threads_started":1}
I20260809 12:09:26.982456 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:26.985630 530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7: Bootstrap starting.
I20260809 12:09:26.988910 530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:26.989686 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:26.986913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55305 (local address 127.0.147.126:46639)
0809 12:09:26.987140 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:26.987149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:26.987724 (+ 575us) server_negotiation.cc:407] Connection header received
0809 12:09:26.987912 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:26.987918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:26.987974 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:26.988084 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:26.989001 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:26.989006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:26.989011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:09:26.989034 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:26.989090 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:09:26.989109 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:26.989241 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:26.989338 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 12:09:26.989401 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 12:09:26.990108 530 log.cc:824] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:26.992884 530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7: No bootstrap required, opened a new log
I20260809 12:09:26.999994 530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2beb98234e6a4a01843a0679596892d7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46639 } }
I20260809 12:09:27.000384 530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:09:27.000492 530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2beb98234e6a4a01843a0679596892d7, State: Initialized, Role: FOLLOWER
I20260809 12:09:27.001068 530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2beb98234e6a4a01843a0679596892d7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46639 } }
I20260809 12:09:27.001309 530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:09:27.001461 530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:09:27.001639 530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:27.003801 530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2beb98234e6a4a01843a0679596892d7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46639 } }
I20260809 12:09:27.004288 530 leader_election.cc:302] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2beb98234e6a4a01843a0679596892d7; no voters:
I20260809 12:09:27.004854 530 leader_election.cc:288] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:09:27.004997 544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:27.006206 544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [term 1 LEADER]: Becoming Leader. State: Replica: 2beb98234e6a4a01843a0679596892d7, State: Running, Role: LEADER
I20260809 12:09:27.006863 544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2beb98234e6a4a01843a0679596892d7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46639 } }
I20260809 12:09:27.007194 530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:27.010854 547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2beb98234e6a4a01843a0679596892d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2beb98234e6a4a01843a0679596892d7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46639 } } }
I20260809 12:09:27.011235 547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:27.011626 548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2beb98234e6a4a01843a0679596892d7. Latest consensus state: current_term: 1 leader_uuid: "2beb98234e6a4a01843a0679596892d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2beb98234e6a4a01843a0679596892d7" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 46639 } } }
I20260809 12:09:27.011967 548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:27.013623 561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:27.019030 561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:09:27.028044 561 catalog_manager.cc:1347] Generated new cluster ID: c18c6b9c1be143e8b98f7981a5176c9e
I20260809 12:09:27.028221 561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:27.052718 561 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:27.053164 561 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:27.061336 561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2beb98234e6a4a01843a0679596892d7: Generated new TSK 0
I20260809 12:09:27.062117 561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:27.072003 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 432
W20260809 12:09:27.073154 524 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39155
W20260809 12:09:27.073868 589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 432: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:09:27.074018 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 432
2026-08-09T12:09:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2235 ms)
[----------] 18 tests from AuthTokenIssuingTest (40773 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:27 dist-test-slave-sb2z krb5kdc[581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:27 dist-test-slave-sb2z krb5kdc[581](info): set up 2 sockets
Aug 09 12:09:27 dist-test-slave-sb2z krb5kdc[581](info): commencing operation
krb5kdc: starting...
W20260809 12:09:28.434190 589 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.001s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:28 dist-test-slave-sb2z krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:09:28.791263 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:44771
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:44771,127.0.147.125:39379,127.0.147.124:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:29.138062 605 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:09:29.138454 605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:29.138590 605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:29.138731 605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:29.146039 605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:29.146160 605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:29.146278 605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:29.146364 605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:29.146456 605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:29.156101 605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:44771,127.0.147.125:39379,127.0.147.124:41181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:44771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:29.157375 605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:29.158726 605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:29.170912 613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:29.170945 614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:29.171110 619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:29.171743 605 server_base.cc:1034] running on GCE node
I20260809 12:09:29.172586 605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:29.173924 605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:29.175180 605 hybrid_clock.cc:648] HybridClock initialized: now 1786277369175117 us; error 41 us; skew 500 ppm
I20260809 12:09:29.175565 605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:29 dist-test-slave-sb2z krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:29.178789 605 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:09:29.180299 605 webserver.cc:460] Webserver started at http://127.0.147.126:36829/ using document root <none> and password file <none>
I20260809 12:09:29.180989 605 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:29.181131 605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:29.181454 605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:29.184448 605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "2ffce010e3ab4b259bb912f3835d1298"
format_stamp: "Formatted at 2026-08-09 12:09:29 on dist-test-slave-sb2z"
server_key: "3d158bff226c87551e33497d1fc7a363"
server_key_iv: "a94f0437550ef39e79c26e5472103902"
server_key_version: "encryptionkey@0"
I20260809 12:09:29.185236 605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "2ffce010e3ab4b259bb912f3835d1298"
format_stamp: "Formatted at 2026-08-09 12:09:29 on dist-test-slave-sb2z"
server_key: "3d158bff226c87551e33497d1fc7a363"
server_key_iv: "a94f0437550ef39e79c26e5472103902"
server_key_version: "encryptionkey@0"
I20260809 12:09:29.190308 605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:09:29.193748 628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:29.194983 605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:09:29.195176 605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "2ffce010e3ab4b259bb912f3835d1298"
format_stamp: "Formatted at 2026-08-09 12:09:29 on dist-test-slave-sb2z"
server_key: "3d158bff226c87551e33497d1fc7a363"
server_key_iv: "a94f0437550ef39e79c26e5472103902"
server_key_version: "encryptionkey@0"
I20260809 12:09:29.195390 605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:29.214887 605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:29.219074 605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:29.219388 605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:29.239487 605 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:44771
I20260809 12:09:29.239588 701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:44771 every 8 connection(s)
I20260809 12:09:29.240896 605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:29.244647 702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:29.246199 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 605
I20260809 12:09:29.246452 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:29.247305 589 external_mini_cluster.cc:1428] Setting key 173fa1d50846ad7f3419635735ed8949
I20260809 12:09:29.254297 702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:09:29 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:09:29.281122 705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.258152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56412 (local address 127.0.147.126:44771)
0809 12:09:29.260011 (+ 1859us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:29.260017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:29.260061 (+ 44us) server_negotiation.cc:407] Connection header received
0809 12:09:29.262340 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:29.262345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:29.262572 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:29.262738 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:29.264399 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.266391 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.267854 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.268845 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.269040 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:29.272623 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:29.272636 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:29.272654 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:29.272702 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:29.276805 (+ 4103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:29.277533 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:29.277544 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:29.277550 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:29.277756 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:29.278212 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:29.278220 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:29.278226 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:29.278947 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:29.279170 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:29.279738 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 12:09:29.280139 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":1659,"spinlock_wait_cycles":25984,"thread_start_us":1430,"threads_started":1}
I20260809 12:09:29.281144 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.258152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:44771 (local address 127.0.0.1:56412)
0809 12:09:29.259064 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.259132 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:29.261979 (+ 2847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:29.262811 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:29.262821 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:29.263689 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:29.264176 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:29.264195 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.266560 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.266564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:29.267699 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:29.267712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.269222 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.269245 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:29.269406 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:29.270084 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:29.270107 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:29.272359 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:29.276975 (+ 4616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:29.276984 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:29.276993 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:29.277376 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:29.277869 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:29.277872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:29.277874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:29.278064 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:29.279097 (+ 1033us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:29.279102 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:29.279286 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 12:09:29.279490 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 12:09:29.279919 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":369,"threads_started":1}
I20260809 12:09:29.284341 702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:09:29.285530 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.285136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:34962)
0809 12:09:29.285305 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.285420 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.0.147.125@KRBTEST.COM; defaulting to no policy
W20260809 12:09:29.285866 630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
I20260809 12:09:29.287351 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.287019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:34976)
0809 12:09:29.287166 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.287234 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:09:29.287639 702 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:39379: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab.
Entry for principal kudu/127.0.147.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab.
I20260809 12:09:29.331264 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.125:39379
--webserver_interface=127.0.147.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab
--principal=kudu/127.0.147.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:44771,127.0.147.125:39379,127.0.147.124:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:09:29.336987 702 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } attempt: 1
I20260809 12:09:29.338200 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.337738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:34988)
0809 12:09:29.337942 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.338066 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 12:09:29.339298 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.338958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35000)
0809 12:09:29.339101 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.339167 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 12:09:29.339592 702 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:39379: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
I20260809 12:09:29.407821 702 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } attempt: 2
I20260809 12:09:29.409104 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.408603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35006)
0809 12:09:29.408854 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.408957 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 12:09:29.410198 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.409795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35014)
0809 12:09:29.409971 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.410065 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:09:29.410423 702 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:39379: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
I20260809 12:09:29.501657 702 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } attempt: 3
I20260809 12:09:29.502874 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.502423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35026)
0809 12:09:29.502644 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.502739 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 12:09:29.504019 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.503663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35038)
0809 12:09:29.503811 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.503880 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 12:09:29.504221 702 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:39379: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
I20260809 12:09:29.647487 702 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } attempt: 4
I20260809 12:09:29.648985 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.648409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35054)
0809 12:09:29.648739 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.648844 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260809 12:09:29.650079 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.649715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35058)
0809 12:09:29.649871 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.649961 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 12:09:29.650259 702 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:39379: Network error: Client connection negotiation failed: client connection to 127.0.147.125:39379: connect: Connection refused (error 111)
W20260809 12:09:29.680277 713 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:09:29.680650 713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:29.680776 713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:29.680932 713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:29.688176 713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:29.688282 713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:29.688387 713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:29.688472 713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:29.688570 713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:29.698021 713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:44771,127.0.147.125:39379,127.0.147.124:41181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab
--principal=kudu/127.0.147.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.125:39379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.147.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:29.699314 713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:29.700749 713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:29.712352 724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:29.712383 725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:29.712950 728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:29.713209 713 server_base.cc:1034] running on GCE node
I20260809 12:09:29.714059 713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:29.715394 713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:29.716651 713 hybrid_clock.cc:648] HybridClock initialized: now 1786277369716578 us; error 41 us; skew 500 ppm
I20260809 12:09:29.716981 713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:29 dist-test-slave-sb2z krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:29.720189 713 init.cc:377] Logged in from keytab as kudu/127.0.147.125@KRBTEST.COM (short username kudu)
I20260809 12:09:29.721668 713 webserver.cc:460] Webserver started at http://127.0.147.125:35091/ using document root <none> and password file <none>
I20260809 12:09:29.722352 713 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:29.722488 713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:29.722810 713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:29.725740 713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/data/instance:
uuid: "9ef53c1eb37b454b98e7935c4d3b4909"
format_stamp: "Formatted at 2026-08-09 12:09:29 on dist-test-slave-sb2z"
server_key: "5a34d2f1d1e24ea93a1f3e84cb476c61"
server_key_iv: "17b3f2ad17631d6c2ca5ab33b4c3d6f6"
server_key_version: "encryptionkey@0"
I20260809 12:09:29.726526 713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/wal/instance:
uuid: "9ef53c1eb37b454b98e7935c4d3b4909"
format_stamp: "Formatted at 2026-08-09 12:09:29 on dist-test-slave-sb2z"
server_key: "5a34d2f1d1e24ea93a1f3e84cb476c61"
server_key_iv: "17b3f2ad17631d6c2ca5ab33b4c3d6f6"
server_key_version: "encryptionkey@0"
I20260809 12:09:29.731676 713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 12:09:29.735054 739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:29.736460 713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:09:29.736649 713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/wal
uuid: "9ef53c1eb37b454b98e7935c4d3b4909"
format_stamp: "Formatted at 2026-08-09 12:09:29 on dist-test-slave-sb2z"
server_key: "5a34d2f1d1e24ea93a1f3e84cb476c61"
server_key_iv: "17b3f2ad17631d6c2ca5ab33b4c3d6f6"
server_key_version: "encryptionkey@0"
I20260809 12:09:29.736879 713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:29.763633 713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:29.767911 713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:29.768209 713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:29.787629 713 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.125:39379
I20260809 12:09:29.787653 797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.125:39379 every 8 connection(s)
I20260809 12:09:29.789000 713 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
I20260809 12:09:29.792685 798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:29.796356 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 713
I20260809 12:09:29.796561 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-1/wal/instance
I20260809 12:09:29.797138 589 external_mini_cluster.cc:1428] Setting key 701ef8dbfbc86483103514aee16d464b
I20260809 12:09:29.801352 798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:09:29 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:09:29.823695 800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.804684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.0.147.126:44771)
0809 12:09:29.805215 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:29.805224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:29.806207 (+ 983us) server_negotiation.cc:407] Connection header received
0809 12:09:29.808149 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:29.808157 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:29.808250 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:29.808381 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:29.809892 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.811027 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.812550 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.813282 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.813425 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:29.817655 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:29.817659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:29.817662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:29.817706 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:29.820414 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:29.821219 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:29.821224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:29.821230 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:29.821354 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:29.821871 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:29.821875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:29.821877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:29.822144 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:29.822301 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:29.823063 (+ 762us) server_negotiation.cc:299] Negotiation successful
0809 12:09:29.823276 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":246,"threads_started":1}
I20260809 12:09:29.824579 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.805232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:44771 (local address 127.0.0.1:56424)
0809 12:09:29.806024 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.806080 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:29.807886 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:29.808586 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:29.808600 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:29.809104 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:29.809670 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:29.809688 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.811176 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.811180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:29.812407 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:29.812419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.813553 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.813559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:29.813780 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:29.814580 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:29.814617 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:29.817431 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:29.820589 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:29.820597 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:29.820611 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:29.821087 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:29.821470 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:29.821475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:29.821479 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:29.821741 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:29.822277 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:29.822285 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:29.822630 (+ 345us) client_negotiation.cc:769] Sending connection context
0809 12:09:29.822956 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 12:09:29.823454 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":349,"threads_started":1}
I20260809 12:09:29.826220 798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.147.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.124@KRBTEST.COM" created.
Aug 09 12:09:29 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
I20260809 12:09:29.844748 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.827039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35072)
0809 12:09:29.827233 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.827258 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:29.827340 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:29.828454 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:29.828458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:29.828894 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:29.829022 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:29.829030 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.830532 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.830537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:29.831376 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:29.831387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.832367 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.832372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:29.832570 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:29.833161 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:29.833177 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:29.837788 (+ 4611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:29.842333 (+ 4545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:29.842337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:29.842340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:29.842708 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:29.843131 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:29.843135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:29.843137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:29.843261 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:29.844072 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:29.844075 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:29.844185 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 12:09:29.844294 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 12:09:29.844488 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 12:09:29.845247 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.827378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35072 (local address 127.0.147.125:39379)
0809 12:09:29.828073 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:29.828079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:29.828104 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:09:29.828147 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:29.828152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:29.828306 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:29.828471 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:29.829192 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.830389 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.831511 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.832198 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.832362 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:29.838025 (+ 5663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:29.838042 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:29.838061 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:29.838124 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:29.842211 (+ 4087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:29.842843 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:29.842859 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:29.842865 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:29.843015 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:29.843364 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:29.843371 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:29.843377 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:29.843904 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:29.844133 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:29.844474 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 12:09:29.844759 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":402,"threads_started":1}
I20260809 12:09:29.847483 798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:09:29.848773 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.848214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39280)
0809 12:09:29.848407 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.848572 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 12:09:29.849030 741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
I20260809 12:09:29.850652 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.850234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39282)
0809 12:09:29.850409 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.850513 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 12:09:29.850885 798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:41181: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab.
Entry for principal kudu/127.0.147.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab.
I20260809 12:09:29.878065 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.124:41181
--webserver_interface=127.0.147.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab
--principal=kudu/127.0.147.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:44771,127.0.147.125:39379,127.0.147.124:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:09:29.900228 798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } attempt: 1
I20260809 12:09:29.901439 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.900955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39292)
0809 12:09:29.901180 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.901282 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 12:09:29.902631 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.902151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39296)
0809 12:09:29.902337 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.902462 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 12:09:29.902858 798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:41181: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
I20260809 12:09:29.949491 702 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } attempt: 5
Aug 09 12:09:29 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
I20260809 12:09:29.963821 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.950298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35080)
0809 12:09:29.950531 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.950555 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:29.950697 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:29.951104 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:29.951109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:29.951610 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:29.951805 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:29.951814 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.952807 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.952814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:29.953690 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:29.953701 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.954471 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.954477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:29.954710 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:29.955357 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:29.955374 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:29.957522 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:29.961470 (+ 3948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:29.961474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:29.961477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:29.961842 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:29.962250 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:29.962254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:29.962256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:29.962387 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:29.963047 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:29.963050 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:29.963178 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 12:09:29.963293 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 12:09:29.963510 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 12:09:29.964032 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.950401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35080 (local address 127.0.147.125:39379)
0809 12:09:29.950582 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:29.950589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:29.950616 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:09:29.950853 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:29.950857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:29.950938 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:29.951072 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:29.951964 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.952668 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.953847 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:29.954318 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:29.954460 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:29.957729 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:29.957736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:29.957742 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:29.957818 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:29.961313 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:29.961969 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:29.961975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:29.961978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:29.962135 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:29.962504 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:29.962511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:29.962516 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:29.962916 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:29.963095 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:29.963454 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 12:09:29.963712 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:09:29.965135 702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:09:29.966266 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.965851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39306)
0809 12:09:29.966052 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.966129 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 12:09:29.967351 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.966920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39316)
0809 12:09:29.967075 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.967184 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 12:09:29.967622 702 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:41181: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
I20260809 12:09:29.971083 798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } attempt: 2
I20260809 12:09:29.972102 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.971656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39330)
0809 12:09:29.971836 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.971937 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 12:09:29.973101 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:29.972738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39344)
0809 12:09:29.972882 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:29.972965 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 12:09:29.973310 798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:41181: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
I20260809 12:09:30.018899 702 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } attempt: 1
I20260809 12:09:30.020217 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.019654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39360)
0809 12:09:30.019875 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.019995 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 12:09:30.021267 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.020884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39368)
0809 12:09:30.021053 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.021137 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 12:09:30.021497 702 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:41181: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
I20260809 12:09:30.064555 798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } attempt: 3
I20260809 12:09:30.065850 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.065240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39372)
0809 12:09:30.065525 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.065646 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 12:09:30.067015 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.066545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39374)
0809 12:09:30.066781 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.066876 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260809 12:09:30.067272 798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:41181: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
I20260809 12:09:30.089772 702 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } attempt: 2
I20260809 12:09:30.091287 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.090580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39378)
0809 12:09:30.090865 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.091034 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 12:09:30.092625 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.092133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39388)
0809 12:09:30.092349 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.092465 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 12:09:30.092880 702 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:41181: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
I20260809 12:09:30.199242 702 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } attempt: 3
I20260809 12:09:30.200568 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.200016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39400)
0809 12:09:30.200254 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.200360 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 12:09:30.201823 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.201349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39406)
0809 12:09:30.201589 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.201685 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260809 12:09:30.202153 702 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:41181: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
I20260809 12:09:30.210496 798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } attempt: 4
I20260809 12:09:30.211654 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.211159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39410)
0809 12:09:30.211362 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.211443 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 12:09:30.212653 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.212327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39414)
0809 12:09:30.212485 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.212557 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 12:09:30.212924 798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:41181: Network error: Client connection negotiation failed: client connection to 127.0.147.124:41181: connect: Connection refused (error 111)
W20260809 12:09:30.249019 807 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:09:30.249353 807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:30.249475 807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:30.249609 807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:30.257026 807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:30.257108 807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:30.257378 807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:30.257493 807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:30.257603 807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:30.268790 807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:44771,127.0.147.125:39379,127.0.147.124:41181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab
--principal=kudu/127.0.147.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.124:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.147.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:30.270164 807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:30.271661 807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:30.283669 815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:30.283609 818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:30.283638 820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:30.284768 807 server_base.cc:1034] running on GCE node
I20260809 12:09:30.285616 807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:30.287005 807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:30.288254 807 hybrid_clock.cc:648] HybridClock initialized: now 1786277370288193 us; error 54 us; skew 500 ppm
I20260809 12:09:30.288600 807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:30 dist-test-slave-sb2z krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:30.291873 807 init.cc:377] Logged in from keytab as kudu/127.0.147.124@KRBTEST.COM (short username kudu)
I20260809 12:09:30.293330 807 webserver.cc:460] Webserver started at http://127.0.147.124:32955/ using document root <none> and password file <none>
I20260809 12:09:30.294015 807 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:30.294152 807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:30.294467 807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:30.297494 807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/data/instance:
uuid: "a177335c0748444dbf52a4e786c67d07"
format_stamp: "Formatted at 2026-08-09 12:09:30 on dist-test-slave-sb2z"
server_key: "aa21949da4b10c85052ec1a563fcfe5f"
server_key_iv: "44762ba1ce46ae514283c6ca37d04f6c"
server_key_version: "encryptionkey@0"
I20260809 12:09:30.298307 807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/wal/instance:
uuid: "a177335c0748444dbf52a4e786c67d07"
format_stamp: "Formatted at 2026-08-09 12:09:30 on dist-test-slave-sb2z"
server_key: "aa21949da4b10c85052ec1a563fcfe5f"
server_key_iv: "44762ba1ce46ae514283c6ca37d04f6c"
server_key_version: "encryptionkey@0"
I20260809 12:09:30.303578 807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:09:30.306903 830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:30.308284 807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:09:30.308475 807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/wal
uuid: "a177335c0748444dbf52a4e786c67d07"
format_stamp: "Formatted at 2026-08-09 12:09:30 on dist-test-slave-sb2z"
server_key: "aa21949da4b10c85052ec1a563fcfe5f"
server_key_iv: "44762ba1ce46ae514283c6ca37d04f6c"
server_key_version: "encryptionkey@0"
I20260809 12:09:30.308701 807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:30.321909 807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:30.325649 807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:30.325920 807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:30.344480 807 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.124:41181
I20260809 12:09:30.344515 900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.124:41181 every 8 connection(s)
I20260809 12:09:30.345780 807 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
I20260809 12:09:30.349402 904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:30.352869 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 807
I20260809 12:09:30.353081 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786276990256303-589-0/minicluster-data/master-2/wal/instance
I20260809 12:09:30.353643 589 external_mini_cluster.cc:1428] Setting key 800bbeb78e9b26af2f04eb8f49d6d475
I20260809 12:09:30.357172 904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:09:30 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
Aug 09 12:09:30 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:09:30.377282 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.360814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56436 (local address 127.0.147.126:44771)
0809 12:09:30.362145 (+ 1331us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.362151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.362170 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:09:30.362250 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.362255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.362326 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.362439 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.363668 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.364360 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.365840 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.366598 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.366750 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.370326 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.370333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.370338 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.370395 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.373988 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.374783 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.374787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.374789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.374913 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.375328 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.375333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.375336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.375716 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.375878 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.376645 (+ 767us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.376875 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1211,"thread_start_us":1097,"threads_started":1}
I20260809 12:09:30.378123 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.355613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.0.147.124:41181)
0809 12:09:30.356427 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.356438 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.356482 (+ 44us) server_negotiation.cc:407] Connection header received
0809 12:09:30.358056 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.358085 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.358486 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.358831 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.360774 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.362790 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.364190 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.364859 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.365101 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.369867 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.369877 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.369888 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.369922 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.373611 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.374400 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.374407 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.374409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.374580 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.375006 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.375013 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.375016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.375742 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.376137 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.376415 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.376899 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":609,"thread_start_us":374,"threads_started":1}
I20260809 12:09:30.378144 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.360701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:44771 (local address 127.0.0.1:56436)
0809 12:09:30.361480 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.361532 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:30.361700 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:30.362512 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:30.362516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:30.363206 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:30.363460 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.363468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.364529 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.364533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.365667 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.365683 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.366854 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.366860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.366983 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.367622 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:30.367648 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:30.370127 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:30.374121 (+ 3994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.374132 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.374146 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.374659 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.375070 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.375074 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.375076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.375214 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.375875 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:30.375882 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:30.376218 (+ 336us) client_negotiation.cc:769] Sending connection context
0809 12:09:30.376488 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 12:09:30.377067 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":141,"thread_start_us":356,"threads_started":1}
I20260809 12:09:30.379396 702 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } attempt: 4
I20260809 12:09:30.380584 904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:09:30 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
Aug 09 12:09:30 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
Aug 09 12:09:30 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:09:30.397253 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.379987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39436)
0809 12:09:30.380214 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.380233 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:30.380346 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:30.381041 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:30.381045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:30.381555 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:30.381791 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.381803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.383447 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.383455 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.384949 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.384970 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.385994 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.386002 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.386224 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.386842 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:30.386860 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:30.390241 (+ 3381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:30.394155 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.394160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.394164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.394777 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.395258 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.395264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.395268 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.395473 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.396184 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:30.396193 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:30.396361 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 12:09:30.396548 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 12:09:30.396838 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 12:09:30.398147 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.380487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39436 (local address 127.0.147.124:41181)
0809 12:09:30.380656 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.380662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.380679 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:09:30.380746 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.380753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.380829 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.381946 (+ 1117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.382119 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.383275 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.385159 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.385831 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.385982 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.390454 (+ 4472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.390460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.390466 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.390503 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.394004 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.394927 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.394934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.394937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.395122 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.395647 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.395653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.395658 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.396026 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.396185 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.397570 (+ 1385us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.397847 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:09:30.399042 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.381831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:39379 (local address 127.0.0.1:35092)
0809 12:09:30.382511 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.382529 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:30.382630 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:30.383099 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:30.383104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:30.383740 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:30.384049 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.384066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.385596 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.385602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.387060 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.387077 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.387858 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.387864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.388066 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.388646 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:30.388671 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:30.391902 (+ 3231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:30.396380 (+ 4478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.396386 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.396389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.396766 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.397163 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.397168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.397171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.397336 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.398014 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:30.398019 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:30.398179 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 12:09:30.398358 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 12:09:30.398676 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569}
I20260809 12:09:30.399122 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.382195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35092 (local address 127.0.147.125:39379)
0809 12:09:30.382375 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.382381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.382592 (+ 211us) server_negotiation.cc:407] Connection header received
0809 12:09:30.382731 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.382736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.382817 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.382916 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.384244 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.385440 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.387186 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.387697 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.387837 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.392126 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.392134 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.392141 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.392194 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.396246 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.396893 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.396906 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.396910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.397049 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.397499 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.397503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.397506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.397865 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.398031 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.398550 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.398766 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:09:30.400378 904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:09:30.400995 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.382307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.0.147.126:44771)
0809 12:09:30.382461 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.382466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.382487 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:09:30.382728 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.382734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.382812 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.382917 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.384663 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.385387 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.387109 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.387909 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.388053 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.393876 (+ 5823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.393882 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.393888 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.393960 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.397583 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.398491 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.398498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.398501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.398650 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.399157 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.399163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.399167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.399582 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.399861 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.400423 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.400648 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:09:30.405278 702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298: Bootstrap starting.
I20260809 12:09:30.409229 702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:30.410673 702 log.cc:824] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:09:30 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
Aug 09 12:09:30 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
I20260809 12:09:30.414695 702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298: No bootstrap required, opened a new log
I20260809 12:09:30.418741 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.401401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39444)
0809 12:09:30.403688 (+ 2287us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.403705 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:30.403829 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:30.404173 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:30.404177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:30.404587 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:30.404720 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.404729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.406285 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.406290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.407693 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.407708 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.408751 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.408758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.409027 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.409591 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:30.409609 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:30.412042 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:30.415871 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.415880 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.415884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.416282 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.416699 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.416705 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.416709 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.416950 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.417545 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:30.417550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:30.417696 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 12:09:30.417946 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 12:09:30.418274 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2189,"mutex_wait_us":25}
I20260809 12:09:30.418766 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.401293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39444 (local address 127.0.147.124:41181)
0809 12:09:30.401481 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.401486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.403776 (+ 2290us) server_negotiation.cc:407] Connection header received
0809 12:09:30.403962 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.403968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.404036 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.404157 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.404862 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.406118 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.407863 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.408589 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.408741 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.412270 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.412277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.412281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.412318 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.415739 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.416431 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.416436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.416442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.416571 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.417102 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.417108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.417111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.417408 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.417591 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.418078 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.418405 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":150,"server-negotiator.queue_time_us":56}
I20260809 12:09:30.422286 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.402888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35094 (local address 127.0.147.125:39379)
0809 12:09:30.403058 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.403062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.403079 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:09:30.403225 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.403228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.403284 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.403367 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.405039 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.406180 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.407521 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.408182 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.408320 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.414450 (+ 6130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.414455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.414460 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.414496 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.418360 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.419261 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.419267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.419271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.419444 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.419952 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.419959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.419962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.420311 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.420588 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.421709 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.421941 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:09:30.424077 904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07: Bootstrap starting.
I20260809 12:09:30.426326 702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } }
I20260809 12:09:30.426805 702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ffce010e3ab4b259bb912f3835d1298, State: Initialized, Role: FOLLOWER
I20260809 12:09:30.427397 702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } }
I20260809 12:09:30.427775 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:30.427980 904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:30.428541 911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.429036 911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:30.429288 904 log.cc:824] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:30.429795 702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:30.431329 663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2ffce010e3ab4b259bb912f3835d1298"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56446
I20260809 12:09:30.431877 663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 12:09:30.432080 663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:30.432876 904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07: No bootstrap required, opened a new log
I20260809 12:09:30.435480 663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } }
I20260809 12:09:30.437793 663 leader_election.cc:288] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [CANDIDATE]: Term 1 election: Requested vote from peers 9ef53c1eb37b454b98e7935c4d3b4909 (127.0.147.125:39379), a177335c0748444dbf52a4e786c67d07 (127.0.147.124:41181)
I20260809 12:09:30.445290 904 raft_consensus.cc:348] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } }
I20260809 12:09:30.445972 904 raft_consensus.cc:729] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a177335c0748444dbf52a4e786c67d07, State: Initialized, Role: FOLLOWER
I20260809 12:09:30.446712 904 consensus_queue.cc:260] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } }
W20260809 12:09:30.456228 927 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:09:30.456383 927 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:09:30.457346 930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.457757 930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:30.459843 904 sys_catalog.cc:564] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:30.459954 863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ffce010e3ab4b259bb912f3835d1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a177335c0748444dbf52a4e786c67d07"
I20260809 12:09:30.460453 863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:30.452276 605 master_runner.cc:186] Error getting master registration for 127.0.147.125:39379: OK, instance_id { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" instance_seqno: 1786277369782178 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:09:30.463011 605 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:09:30.464089 863 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ffce010e3ab4b259bb912f3835d1298 in term 1.
I20260809 12:09:30.464892 631 leader_election.cc:302] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ffce010e3ab4b259bb912f3835d1298, a177335c0748444dbf52a4e786c67d07; no voters:
I20260809 12:09:30.465428 911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:30.467609 911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [term 1 LEADER]: Becoming Leader. State: Replica: 2ffce010e3ab4b259bb912f3835d1298, State: Running, Role: LEADER
I20260809 12:09:30.468233 911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } }
W20260809 12:09:30.472460 636 tablet.cc:2369] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:09:30.512135 798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } attempt: 5
I20260809 12:09:30.522007 937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ffce010e3ab4b259bb912f3835d1298. Latest consensus state: current_term: 1 leader_uuid: "2ffce010e3ab4b259bb912f3835d1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.522975 937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:30.540171 949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:30.541504 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.478291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35108 (local address 127.0.147.125:39379)
0809 12:09:30.478534 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.478542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.512355 (+ 33813us) server_negotiation.cc:407] Connection header received
0809 12:09:30.512757 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.512763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.512838 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.512914 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.525907 (+ 12993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.527098 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.529110 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.529787 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.529912 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.533747 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.533754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.533759 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.533811 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.537575 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.538622 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.538627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.538630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.538813 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.539306 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.539312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.539315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.539700 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.540007 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.540746 (+ 739us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.541082 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:09:30.543313 949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 12:09:30 dist-test-slave-sb2z krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
I20260809 12:09:30.576673 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.488193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.0.147.124:41181)
0809 12:09:30.488375 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.488383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.535549 (+ 47166us) server_negotiation.cc:407] Connection header received
0809 12:09:30.535875 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.535884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.536022 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.536114 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.555808 (+ 19694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.556915 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.559263 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.559966 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.560141 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.566329 (+ 6188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.566337 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.566342 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.566385 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.570987 (+ 4602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.572445 (+ 1458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.572453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.572457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.572731 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.573499 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.573531 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.573537 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.574199 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.574613 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.575558 (+ 945us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.575967 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 12:09:30.578486 837 tablet.cc:2369] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:09:30.580127 957 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 12:09:30.587620 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.512823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:41181 (local address 127.0.0.1:39456)
0809 12:09:30.513172 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:30.513191 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:30.513303 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:30.548723 (+ 35420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:30.548728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:30.549249 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:30.549466 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.549478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.553025 (+ 3547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.553031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.554437 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:30.554453 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.561744 (+ 7291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.561753 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:30.562106 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.562905 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:30.562937 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:30.566617 (+ 3680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:30.584234 (+ 17617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.584242 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.584248 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.584918 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.585423 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:30.585431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:30.585434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:30.585644 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:30.586481 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:30.586487 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:30.586672 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 12:09:30.586832 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 12:09:30.587162 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236}
I20260809 12:09:30.588901 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.513146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39456 (local address 127.0.147.124:41181)
0809 12:09:30.548203 (+ 35057us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.548212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.548246 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:09:30.548323 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.548332 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.548446 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.548673 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.551201 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.552869 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.555850 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.556868 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.566857 (+ 9989us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.567177 (+ 320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.567185 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.567192 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.567233 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.575114 (+ 7881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.585097 (+ 9983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.585102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.585105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.585265 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.585935 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.585940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.585943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.586337 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.586482 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.587131 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.587729 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":34846,"thread_start_us":42386,"threads_started":1}
I20260809 12:09:30.594200 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:30.476133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.0.147.126:44771)
0809 12:09:30.476361 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.476371 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.506517 (+ 30146us) server_negotiation.cc:407] Connection header received
0809 12:09:30.506915 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.506921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.507024 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.507125 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.549594 (+ 42469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.550952 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.559433 (+ 8481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:30.560370 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:30.560500 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:30.575446 (+ 14946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:30.575457 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:30.575464 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:30.575556 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:30.581485 (+ 5929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.582398 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.582403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.582407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.582583 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:30.583080 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:30.583085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:30.583088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:30.583851 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:30.593348 (+ 9497us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:30.593498 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 12:09:30.593806 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 12:09:30.588644 807 master_runner.cc:186] Error getting master registration for 127.0.147.125:39379: OK, instance_id { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" instance_seqno: 1786277369782178 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:09:30.600237 807 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:09:30.602438 798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909: Bootstrap starting.
I20260809 12:09:30.607328 798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:30.608664 863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [term 1 FOLLOWER]: Refusing update from remote peer 2ffce010e3ab4b259bb912f3835d1298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:09:30.608741 798 log.cc:824] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:30.609704 937 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:09:30.621991 798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909: No bootstrap required, opened a new log
I20260809 12:09:30.631086 930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ffce010e3ab4b259bb912f3835d1298. Latest consensus state: current_term: 1 leader_uuid: "2ffce010e3ab4b259bb912f3835d1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.631455 930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:30.633105 937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ffce010e3ab4b259bb912f3835d1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.633399 937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:30.634496 961 mvcc.cc:205] Tried to move back new op lower bound from 7316592110000041984 to 7316592109443805184. Current Snapshot: MvccSnapshot[applied={T|T < 7316592110000041984}]
I20260809 12:09:30.635417 930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ffce010e3ab4b259bb912f3835d1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.635694 930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:30.638938 798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } }
I20260809 12:09:30.639372 798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef53c1eb37b454b98e7935c4d3b4909, State: Initialized, Role: FOLLOWER
I20260809 12:09:30.639946 798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } }
I20260809 12:09:30.641027 949 catalog_manager.cc:1347] Generated new cluster ID: 8123bd1e70a644a4a6bf6b10832b4569
I20260809 12:09:30.641161 949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:30.647228 969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.647661 969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:30.650350 798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:30.650779 768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:30.651649 769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ffce010e3ab4b259bb912f3835d1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9ef53c1eb37b454b98e7935c4d3b4909"
I20260809 12:09:30.656488 768 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [term 1 FOLLOWER]: Refusing update from remote peer 2ffce010e3ab4b259bb912f3835d1298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:09:30.657626 911 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:09:30.662752 769 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ffce010e3ab4b259bb912f3835d1298 in term 1.
I20260809 12:09:30.682334 949 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:30.684177 949 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:30.698597 969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ffce010e3ab4b259bb912f3835d1298. Latest consensus state: current_term: 1 leader_uuid: "2ffce010e3ab4b259bb912f3835d1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.699030 969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:30.701531 911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ffce010e3ab4b259bb912f3835d1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.701923 911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298 [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:30.703087 949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ffce010e3ab4b259bb912f3835d1298: Generated new TSK 0
I20260809 12:09:30.707257 949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:09:30.709102 979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ffce010e3ab4b259bb912f3835d1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffce010e3ab4b259bb912f3835d1298" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 44771 } } peers { permanent_uuid: "9ef53c1eb37b454b98e7935c4d3b4909" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 39379 } } peers { permanent_uuid: "a177335c0748444dbf52a4e786c67d07" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 41181 } } }
I20260809 12:09:30.709604 979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 12:09:30.733969 989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:09:30.734176 989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 12:09:30.849473 909 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.834798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56464 (local address 127.0.147.126:44771)
0809 12:09:30.835057 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.835064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.845912 (+ 10848us) server_negotiation.cc:407] Connection header received
0809 12:09:30.847065 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.847076 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.847188 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.847287 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.849119 (+ 1832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56464: BlockingRecv error: recv got EOF from 127.0.0.1:56464 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 12:09:30.853788 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.835786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.0.147.125:39379)
0809 12:09:30.835967 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.835975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.851862 (+ 15887us) server_negotiation.cc:407] Connection header received
0809 12:09:30.852150 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.852155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.852255 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.852323 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.853545 (+ 1222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35124: BlockingRecv error: recv got EOF from 127.0.0.1:35124 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 12:09:30.853782 948 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.834768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39460 (local address 127.0.147.124:41181)
0809 12:09:30.834945 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.834952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.851795 (+ 16843us) server_negotiation.cc:407] Connection header received
0809 12:09:30.852070 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.852076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.852166 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.852239 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.853551 (+ 1312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39460: BlockingRecv error: recv got EOF from 127.0.0.1:39460 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 12:09:30.863586 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.861326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35134 (local address 127.0.147.125:39379)
0809 12:09:30.861551 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.861560 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.862085 (+ 525us) server_negotiation.cc:407] Connection header received
0809 12:09:30.862327 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.862338 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.862439 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.862566 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.863255 (+ 689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35134: BlockingRecv error: recv got EOF from 127.0.0.1:35134 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 12:09:30.870141 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.868231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.0.147.125:39379)
0809 12:09:30.868445 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.868453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.868912 (+ 459us) server_negotiation.cc:407] Connection header received
0809 12:09:30.869112 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.869118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.869189 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.869294 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.869954 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35144: BlockingRecv error: recv got EOF from 127.0.0.1:35144 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 12:09:30.876953 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.874887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.0.147.125:39379)
0809 12:09:30.875094 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.875102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.875625 (+ 523us) server_negotiation.cc:407] Connection header received
0809 12:09:30.875830 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.875837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.875910 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.876026 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.876720 (+ 694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35146: BlockingRecv error: recv got EOF from 127.0.0.1:35146 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 12:09:30.888679 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.885406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35156 (local address 127.0.147.125:39379)
0809 12:09:30.885705 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.885714 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.887287 (+ 1573us) server_negotiation.cc:407] Connection header received
0809 12:09:30.887452 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.887457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.887621 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.887737 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.888472 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35156: BlockingRecv error: recv got EOF from 127.0.0.1:35156 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
W20260809 12:09:30.901878 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.899949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35170 (local address 127.0.147.125:39379)
0809 12:09:30.900183 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.900191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.900635 (+ 444us) server_negotiation.cc:407] Connection header received
0809 12:09:30.900832 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.900843 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.900918 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.901024 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.901660 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35170: BlockingRecv error: recv got EOF from 127.0.0.1:35170 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 12:09:30.923271 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.920922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.0.147.125:39379)
0809 12:09:30.921139 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.921148 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.921767 (+ 619us) server_negotiation.cc:407] Connection header received
0809 12:09:30.922051 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.922084 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.922152 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.922264 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.922989 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35180: BlockingRecv error: recv got EOF from 127.0.0.1:35180 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 12:09:30.960943 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:30.958567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35196 (local address 127.0.147.125:39379)
0809 12:09:30.958803 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:30.958812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:30.959315 (+ 503us) server_negotiation.cc:407] Connection header received
0809 12:09:30.959564 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:30.959570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:30.959654 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:30.959790 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:30.960644 (+ 854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35196: BlockingRecv error: recv got EOF from 127.0.0.1:35196 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 12:09:31.034770 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:31.032450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35200 (local address 127.0.147.125:39379)
0809 12:09:31.032692 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:31.032701 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:31.033284 (+ 583us) server_negotiation.cc:407] Connection header received
0809 12:09:31.033554 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:31.033563 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:31.033635 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:31.033753 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:31.034526 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35200: BlockingRecv error: recv got EOF from 127.0.0.1:35200 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 12:09:31.173702 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:31.171491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.0.147.125:39379)
0809 12:09:31.171768 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:31.171775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:31.172049 (+ 274us) server_negotiation.cc:407] Connection header received
0809 12:09:31.172280 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:31.172287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:31.172432 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:31.172564 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:31.173394 (+ 830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44472: BlockingRecv error: recv got EOF from 127.0.0.1:44472 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
W20260809 12:09:31.438967 804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:31.437130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44482 (local address 127.0.147.125:39379)
0809 12:09:31.437373 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:31.437380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:31.437526 (+ 146us) server_negotiation.cc:407] Connection header received
0809 12:09:31.437699 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:31.437704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:31.437846 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:31.437937 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:31.438738 (+ 801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44482: BlockingRecv error: recv got EOF from 127.0.0.1:44482 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 12:09:31.584499 957 catalog_manager.cc:1259] Loaded cluster ID: 8123bd1e70a644a4a6bf6b10832b4569
I20260809 12:09:31.584612 957 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07: loading cluster ID for follower catalog manager: success
I20260809 12:09:31.587257 957 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07: acquiring CA information for follower catalog manager: success
I20260809 12:09:31.589143 957 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a177335c0748444dbf52a4e786c67d07: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:09:31.736093 989 catalog_manager.cc:1259] Loaded cluster ID: 8123bd1e70a644a4a6bf6b10832b4569
I20260809 12:09:31.736191 989 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909: loading cluster ID for follower catalog manager: success
I20260809 12:09:31.738775 989 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909: acquiring CA information for follower catalog manager: success
I20260809 12:09:31.740724 989 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9ef53c1eb37b454b98e7935c4d3b4909: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:09:31.960310 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:31.954410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44484 (local address 127.0.147.125:39379)
0809 12:09:31.955064 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:31.955069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:31.955164 (+ 95us) server_negotiation.cc:407] Connection header received
0809 12:09:31.955333 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:31.955337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:31.955498 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:31.955680 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:31.956097 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.956779 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.958082 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.958485 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.958581 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:31.959074 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:31.959660 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:31.959780 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:31.959869 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 12:09:31.959932 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":389,"threads_started":1}
I20260809 12:09:31.972218 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:31.966332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.0.147.124:41181)
0809 12:09:31.966876 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:31.966881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:31.966898 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:09:31.966991 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:31.966995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:31.967152 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:31.967294 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:31.967819 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.968609 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.969893 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.970303 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.970398 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:31.971017 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:31.971625 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:31.971737 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:31.971859 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 12:09:31.971924 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":306,"threads_started":1}
I20260809 12:09:31.988027 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:31.978959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44488 (local address 127.0.147.125:39379)
0809 12:09:31.979183 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:31.979192 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:31.979683 (+ 491us) server_negotiation.cc:407] Connection header received
0809 12:09:31.980080 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:31.980089 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:31.980216 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:31.980293 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:31.981245 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.982675 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.985713 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.986520 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.986652 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:31.987086 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:31.987225 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:31.987314 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:31.987578 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 12:09:31.987672 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 12:09:31.988384 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:31.979546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.0.147.124:41181)
0809 12:09:31.979750 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:31.979758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:31.980251 (+ 493us) server_negotiation.cc:407] Connection header received
0809 12:09:31.980550 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:31.980559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:31.980662 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:31.980734 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:31.981753 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.983163 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.985980 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.986648 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.986764 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:31.987379 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:31.987527 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:31.987612 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:31.988030 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 12:09:31.988113 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 12:09:31.992645 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:31.979244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.0.147.126:44771)
0809 12:09:31.986605 (+ 7361us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:31.986613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:31.986632 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:09:31.986797 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:31.986805 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:31.987032 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:31.987230 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:31.988020 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.989023 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.990665 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:31.991075 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:31.991214 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:31.991711 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:31.992030 (+ 319us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:31.992155 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:31.992307 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 12:09:31.992388 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7225,"thread_start_us":7111,"threads_started":1}
I20260809 12:09:32.000463 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 605
I20260809 12:09:32.001592 692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:32.115829 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 605
W20260809 12:09:32.147660 740 connection.cc:535] server connection from 127.0.0.1:35080 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 12:09:32.148140 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 713
I20260809 12:09:32.149295 793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:32.259268 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 713
I20260809 12:09:32.287745 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 807
I20260809 12:09:32.288893 892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:32.396283 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 807
2026-08-09T12:09:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5324 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:09:32 dist-test-slave-sb2z krb5kdc[1110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:09:32 dist-test-slave-sb2z krb5kdc[1110](info): set up 2 sockets
Aug 09 12:09:32 dist-test-slave-sb2z krb5kdc[1110](info): commencing operation
krb5kdc: starting...
W20260809 12:09:33.732854 589 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/test-user.keytab.
Aug 09 12:09:33 dist-test-slave-sb2z krb5kdc[1110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:09:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:09:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.147.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
Entry for principal kudu/127.0.147.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab.
I20260809 12:09:34.084916 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:34569
--webserver_interface=127.0.147.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:43869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:34569,127.0.147.125:33455,127.0.147.124:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:09:34.433002 1130 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:09:34.433312 1130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:34.433386 1130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:34.433468 1130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:34.440536 1130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:34.440629 1130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:34.440692 1130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:34.440745 1130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:34.440810 1130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:34.450877 1130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:43869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:34569,127.0.147.125:33455,127.0.147.124:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.126.keytab
--principal=kudu/127.0.147.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.126:34569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.147.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:34.452450 1130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:34.453781 1130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:34.465608 1136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:34.466116 1143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:34.466212 1135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:34.466661 1130 server_base.cc:1034] running on GCE node
I20260809 12:09:34.467497 1130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:34.469038 1130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:34.470324 1130 hybrid_clock.cc:648] HybridClock initialized: now 1786277374470253 us; error 41 us; skew 500 ppm
I20260809 12:09:34.470692 1130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:34 dist-test-slave-sb2z krb5kdc[1110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:34.473948 1130 init.cc:377] Logged in from keytab as kudu/127.0.147.126@KRBTEST.COM (short username kudu)
I20260809 12:09:34.475366 1130 webserver.cc:460] Webserver started at http://127.0.147.126:33985/ using document root <none> and password file <none>
I20260809 12:09:34.476099 1130 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:34.476249 1130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:34.476583 1130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:34.479666 1130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/data/instance:
uuid: "e26118b4572847a697ba9ae57e5b006b"
format_stamp: "Formatted at 2026-08-09 12:09:34 on dist-test-slave-sb2z"
server_key: "589649fed9ad77308318fa0719f7c62c"
server_key_iv: "08b3b2a18f7080aa8b8359cea9dbd78b"
server_key_version: "encryptionkey@0"
I20260809 12:09:34.480566 1130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/wal/instance:
uuid: "e26118b4572847a697ba9ae57e5b006b"
format_stamp: "Formatted at 2026-08-09 12:09:34 on dist-test-slave-sb2z"
server_key: "589649fed9ad77308318fa0719f7c62c"
server_key_iv: "08b3b2a18f7080aa8b8359cea9dbd78b"
server_key_version: "encryptionkey@0"
I20260809 12:09:34.485669 1130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:09:34.489048 1151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:34.490278 1130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:09:34.490463 1130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/wal
uuid: "e26118b4572847a697ba9ae57e5b006b"
format_stamp: "Formatted at 2026-08-09 12:09:34 on dist-test-slave-sb2z"
server_key: "589649fed9ad77308318fa0719f7c62c"
server_key_iv: "08b3b2a18f7080aa8b8359cea9dbd78b"
server_key_version: "encryptionkey@0"
I20260809 12:09:34.490693 1130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:34.504050 1130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:34.507817 1130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:34.508095 1130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:34.527195 1130 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.126:34569
I20260809 12:09:34.527215 1221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.126:34569 every 8 connection(s)
I20260809 12:09:34.528656 1130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/data/info.pb
I20260809 12:09:34.529170 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 1130
I20260809 12:09:34.529367 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-0/wal/instance
I20260809 12:09:34.530015 589 external_mini_cluster.cc:1428] Setting key 72bc63d4f3875d1aa932d02d33ddec06
I20260809 12:09:34.532657 1222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:34.539095 1222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:09:34 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:09:34.564724 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.542309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:34569 (local address 127.0.0.1:52436)
0809 12:09:34.543250 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.543308 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:34.546046 (+ 2738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:34.547014 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:34.547018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:34.547667 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:34.547994 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:34.548010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:34.550423 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:34.550427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:34.551462 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:34.551473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:34.552851 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:34.552856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:34.552985 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:34.553699 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:34.553722 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:34.556175 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:34.560619 (+ 4444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:34.560628 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:34.560637 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:34.561054 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:34.561551 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:34.561554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:34.561557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:34.561706 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:34.562723 (+ 1017us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:34.562729 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:34.562902 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 12:09:34.563107 (+ 205us) client_negotiation.cc:240] Negotiation successful
0809 12:09:34.563514 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"spinlock_wait_cycles":4352,"thread_start_us":385,"threads_started":1}
I20260809 12:09:34.564764 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.542309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.0.147.126:34569)
0809 12:09:34.544247 (+ 1938us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:34.544252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:34.544291 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:09:34.546441 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:34.546452 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:34.546766 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:34.547041 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:34.548220 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:34.550259 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:34.551658 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:34.552631 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:34.552832 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:34.556438 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:34.556451 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:34.556469 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:34.556510 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:34.560494 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:34.561224 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:34.561235 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:34.561241 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:34.561442 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:34.561822 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:34.561830 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:34.561836 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:34.562591 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:34.562805 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:34.563347 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 12:09:34.563738 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":1743,"spinlock_wait_cycles":42752,"thread_start_us":1511,"threads_started":1}
I20260809 12:09:34.567775 1222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.147.125@KRBTEST.COM; defaulting to no policy
I20260809 12:09:34.568982 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.568490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37038)
0809 12:09:34.568697 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.568835 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 12:09:34.569212 1164 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
I20260809 12:09:34.570672 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.570342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37052)
0809 12:09:34.570476 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.570545 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:09:34.570880 1222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:33455: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab.
Entry for principal kudu/127.0.147.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab.
I20260809 12:09:34.610795 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.125:33455
--webserver_interface=127.0.147.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab
--principal=kudu/127.0.147.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:43869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:34569,127.0.147.125:33455,127.0.147.124:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:09:34.620256 1222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } attempt: 1
I20260809 12:09:34.621619 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.621167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37054)
0809 12:09:34.621371 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.621491 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 12:09:34.622685 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.622366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37062)
0809 12:09:34.622495 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.622565 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:09:34.622941 1222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:33455: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
I20260809 12:09:34.691246 1222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } attempt: 2
I20260809 12:09:34.692595 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.691996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37072)
0809 12:09:34.692285 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.692403 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 12:09:34.693713 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.693266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37078)
0809 12:09:34.693448 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.693538 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 12:09:34.693949 1222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:33455: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
I20260809 12:09:34.785255 1222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } attempt: 3
I20260809 12:09:34.786450 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.785988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37086)
0809 12:09:34.786221 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.786316 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 12:09:34.787634 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.787223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37096)
0809 12:09:34.787355 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.787423 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:09:34.787894 1222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:33455: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
I20260809 12:09:34.931156 1222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } attempt: 4
I20260809 12:09:34.932528 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.931928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37112)
0809 12:09:34.932288 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.932378 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210}
I20260809 12:09:34.933614 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:34.933280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37118)
0809 12:09:34.933422 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:34.933500 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:09:34.933892 1222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.125:33455: Network error: Client connection negotiation failed: client connection to 127.0.147.125:33455: connect: Connection refused (error 111)
W20260809 12:09:34.955750 1230 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:09:34.956063 1230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:34.956221 1230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:34.956336 1230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:34.963416 1230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:34.963497 1230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:34.963627 1230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:34.963703 1230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:34.963811 1230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:34.973429 1230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:43869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:34569,127.0.147.125:33455,127.0.147.124:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.125.keytab
--principal=kudu/127.0.147.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.125:33455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.147.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:34.974731 1230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:34.976107 1230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:34.987848 1236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:34.987792 1243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:34.987821 1239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:34.989017 1230 server_base.cc:1034] running on GCE node
I20260809 12:09:34.989800 1230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:34.991088 1230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:34.992311 1230 hybrid_clock.cc:648] HybridClock initialized: now 1786277374992247 us; error 33 us; skew 500 ppm
I20260809 12:09:34.992653 1230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:34 dist-test-slave-sb2z krb5kdc[1110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:34.995996 1230 init.cc:377] Logged in from keytab as kudu/127.0.147.125@KRBTEST.COM (short username kudu)
I20260809 12:09:34.997432 1230 webserver.cc:460] Webserver started at http://127.0.147.125:45147/ using document root <none> and password file <none>
I20260809 12:09:34.998070 1230 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:34.998159 1230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:34.998422 1230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:35.001482 1230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/data/instance:
uuid: "216d220a4ec845c18a9212f5a086f432"
format_stamp: "Formatted at 2026-08-09 12:09:34 on dist-test-slave-sb2z"
server_key: "ddef0b26b00cc4515d75e0b32e11577f"
server_key_iv: "2fe42e0377557350caff2f0504d41f30"
server_key_version: "encryptionkey@0"
I20260809 12:09:35.002236 1230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/wal/instance:
uuid: "216d220a4ec845c18a9212f5a086f432"
format_stamp: "Formatted at 2026-08-09 12:09:34 on dist-test-slave-sb2z"
server_key: "ddef0b26b00cc4515d75e0b32e11577f"
server_key_iv: "2fe42e0377557350caff2f0504d41f30"
server_key_version: "encryptionkey@0"
I20260809 12:09:35.007337 1230 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:09:35.010718 1255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:35.012089 1230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:09:35.012272 1230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/wal
uuid: "216d220a4ec845c18a9212f5a086f432"
format_stamp: "Formatted at 2026-08-09 12:09:34 on dist-test-slave-sb2z"
server_key: "ddef0b26b00cc4515d75e0b32e11577f"
server_key_iv: "2fe42e0377557350caff2f0504d41f30"
server_key_version: "encryptionkey@0"
I20260809 12:09:35.012491 1230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:35.025453 1230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:35.029364 1230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:35.029652 1230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:35.048470 1230 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.125:33455
I20260809 12:09:35.048489 1324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.125:33455 every 8 connection(s)
I20260809 12:09:35.049901 1230 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/data/info.pb
I20260809 12:09:35.053503 1325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:35.055260 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 1230
I20260809 12:09:35.055608 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-1/wal/instance
I20260809 12:09:35.056071 589 external_mini_cluster.cc:1428] Setting key f7c5210c9a26ee7b775fca99043b7d55
I20260809 12:09:35.060117 1325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:09:35.084695 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.062854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52442 (local address 127.0.147.126:34569)
0809 12:09:35.063101 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.063109 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.064229 (+ 1120us) server_negotiation.cc:407] Connection header received
0809 12:09:35.067095 (+ 2866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.067100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.067170 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.067263 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.069452 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.070140 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.072607 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.073336 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.073502 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.077239 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.077248 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.077255 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.077303 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.081280 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.082025 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.082034 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.082040 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.082223 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.082652 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.082659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.082665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.083055 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.083219 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.083984 (+ 765us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.084244 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 12:09:35.085336 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.063224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:34569 (local address 127.0.0.1:52442)
0809 12:09:35.064032 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.064089 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:35.066765 (+ 2676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:35.067567 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:35.067583 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:35.068405 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:35.069215 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.069245 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.070326 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.070334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.072425 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.072443 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.073579 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.073584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.073786 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.074516 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:35.074543 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:35.076983 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:35.081415 (+ 4432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.081422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.081433 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.081873 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.082338 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.082341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.082344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.082531 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.083185 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:35.083191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:35.083510 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 12:09:35.083832 (+ 322us) client_negotiation.cc:240] Negotiation successful
0809 12:09:35.084199 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":313,"threads_started":1}
I20260809 12:09:35.087193 1325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.147.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.147.124@KRBTEST.COM" created.
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
I20260809 12:09:35.103835 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.088313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37120 (local address 127.0.147.125:33455)
0809 12:09:35.089014 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.089020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.089046 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:09:35.089087 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.089093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.089255 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.089418 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.090163 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.091341 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.092533 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.093060 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.093208 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.097988 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.097999 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.098012 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.098045 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.100848 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.101705 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.101711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.101713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.101830 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.102254 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.102257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.102259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.102577 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.102732 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.103165 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.103483 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":550,"thread_start_us":387,"threads_started":1}
I20260809 12:09:35.103863 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.087874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37120)
0809 12:09:35.088035 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.088055 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:35.088172 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:35.089410 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:35.089414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:35.089855 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:35.089982 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.089990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.091482 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.091487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.092373 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.092385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.093215 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.093220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.093422 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.094112 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:35.094129 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:35.097780 (+ 3651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:35.101010 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.101017 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.101023 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.101573 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.101938 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.101945 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.101951 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.102145 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.102708 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:35.102715 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:35.102879 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 12:09:35.103042 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 12:09:35.103371 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 12:09:35.106469 1325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:09:35.107640 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.107164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41828)
0809 12:09:35.107333 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.107438 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 12:09:35.107875 1257 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
I20260809 12:09:35.109445 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.109126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41838)
0809 12:09:35.109264 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.109347 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 12:09:35.109681 1325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:39889: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.147.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab.
Entry for principal kudu/127.0.147.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab.
I20260809 12:09:35.138124 589 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3WUkck/build/asan/bin/kudu
/tmp/dist-test-task3WUkck/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.124:39889
--webserver_interface=127.0.147.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab
--principal=kudu/127.0.147.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.147.84:43869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.147.126:34569,127.0.147.125:33455,127.0.147.124:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:09:35.159044 1325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } attempt: 1
I20260809 12:09:35.160390 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.159758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41846)
0809 12:09:35.160022 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.160161 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 12:09:35.161568 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.161120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41848)
0809 12:09:35.161310 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.161415 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:09:35.161785 1325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:39889: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
I20260809 12:09:35.230062 1325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } attempt: 2
I20260809 12:09:35.231310 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.230755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41852)
0809 12:09:35.230992 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.231117 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 12:09:35.232439 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.232049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41864)
0809 12:09:35.232209 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.232300 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 12:09:35.232685 1325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:39889: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
I20260809 12:09:35.233131 1222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } attempt: 5
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
I20260809 12:09:35.247884 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.233771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37126)
0809 12:09:35.233975 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.233999 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:35.234151 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:35.234528 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:35.234533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:35.234951 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:35.235148 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.235157 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.236247 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.236253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.237145 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.237158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.238594 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.238599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.238792 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.239422 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:35.239440 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:35.241502 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:35.245131 (+ 3629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.245136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.245166 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.245770 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.246199 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.246204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.246208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.246396 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.247097 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:35.247101 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:35.247252 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 12:09:35.247368 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 12:09:35.247640 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 12:09:35.248286 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.233787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37126 (local address 127.0.147.125:33455)
0809 12:09:35.233947 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.233953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.234074 (+ 121us) server_negotiation.cc:407] Connection header received
0809 12:09:35.234297 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.234301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.234376 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.234500 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.235295 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.236107 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.237319 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.238106 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.238277 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.241710 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.241715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.241718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.241749 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.244954 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.245887 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.245893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.245897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.246059 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.246566 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.246571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.246575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.246958 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.247121 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.247512 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.247780 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:09:35.249296 1222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:09:35.250314 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.249923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41870)
0809 12:09:35.250088 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.250186 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:09:35.251260 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.250944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41876)
0809 12:09:35.251085 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.251171 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 12:09:35.251490 1222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:39889: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
I20260809 12:09:35.302771 1222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } attempt: 1
I20260809 12:09:35.304009 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.303519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41884)
0809 12:09:35.303755 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.303846 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 12:09:35.305070 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.304710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41890)
0809 12:09:35.304866 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.304964 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 12:09:35.305339 1222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:39889: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
I20260809 12:09:35.323910 1325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } attempt: 3
I20260809 12:09:35.325239 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.324588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41904)
0809 12:09:35.324844 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.324999 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 12:09:35.326695 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.326083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41906)
0809 12:09:35.326390 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.326511 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
W20260809 12:09:35.326982 1325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:39889: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
I20260809 12:09:35.373596 1222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } attempt: 2
I20260809 12:09:35.374856 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.374343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41908)
0809 12:09:35.374594 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.374685 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 12:09:35.376009 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.375691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41916)
0809 12:09:35.375845 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.375908 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 12:09:35.376292 1222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:39889: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
I20260809 12:09:35.470243 1325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } attempt: 4
I20260809 12:09:35.471467 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.470883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41924)
0809 12:09:35.471153 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.471261 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 12:09:35.472581 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.472203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41934)
0809 12:09:35.472371 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.472455 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 12:09:35.472826 1325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:39889: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
I20260809 12:09:35.482507 1222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } attempt: 3
I20260809 12:09:35.483660 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.483121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41946)
0809 12:09:35.483324 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.483404 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 12:09:35.484776 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.484328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41956)
0809 12:09:35.484527 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.484622 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260809 12:09:35.484979 1222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.147.124:39889: Network error: Client connection negotiation failed: client connection to 127.0.147.124:39889: connect: Connection refused (error 111)
W20260809 12:09:35.492851 1336 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:09:35.493189 1336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:09:35.493323 1336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:09:35.493443 1336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:09:35.501030 1336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:09:35.501117 1336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:09:35.501247 1336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:09:35.501318 1336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:09:35.501402 1336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:09:35.511248 1336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.147.84:43869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.147.126:34569,127.0.147.125:33455,127.0.147.124:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/krb5kdc/kudu_127.0.147.124.keytab
--principal=kudu/127.0.147.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.147.124:39889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.147.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:09:35.512667 1336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:09:35.513974 1336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:09:35.525665 1349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:35.525714 1347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:09:35.525719 1345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:09:35.526934 1336 server_base.cc:1034] running on GCE node
I20260809 12:09:35.527787 1336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:09:35.529124 1336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:09:35.530382 1336 hybrid_clock.cc:648] HybridClock initialized: now 1786277375530313 us; error 36 us; skew 500 ppm
I20260809 12:09:35.530714 1336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:09:35.534127 1336 init.cc:377] Logged in from keytab as kudu/127.0.147.124@KRBTEST.COM (short username kudu)
I20260809 12:09:35.535842 1336 webserver.cc:460] Webserver started at http://127.0.147.124:38893/ using document root <none> and password file <none>
I20260809 12:09:35.536561 1336 fs_manager.cc:359] Metadata directory not provided
I20260809 12:09:35.536681 1336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:09:35.537012 1336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:09:35.540567 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/data/instance:
uuid: "eee105ad0b9a427dbcbe792d722a8759"
format_stamp: "Formatted at 2026-08-09 12:09:35 on dist-test-slave-sb2z"
server_key: "40ac26762af89de2c5f259e8cb395195"
server_key_iv: "8f1c4cd17f141b1478eb8cec3b049b0d"
server_key_version: "encryptionkey@0"
I20260809 12:09:35.541404 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/wal/instance:
uuid: "eee105ad0b9a427dbcbe792d722a8759"
format_stamp: "Formatted at 2026-08-09 12:09:35 on dist-test-slave-sb2z"
server_key: "40ac26762af89de2c5f259e8cb395195"
server_key_iv: "8f1c4cd17f141b1478eb8cec3b049b0d"
server_key_version: "encryptionkey@0"
I20260809 12:09:35.546710 1336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:09:35.550094 1356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:09:35.551333 1336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:09:35.551512 1336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/data,/tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/wal
uuid: "eee105ad0b9a427dbcbe792d722a8759"
format_stamp: "Formatted at 2026-08-09 12:09:35 on dist-test-slave-sb2z"
server_key: "40ac26762af89de2c5f259e8cb395195"
server_key_iv: "8f1c4cd17f141b1478eb8cec3b049b0d"
server_key_version: "encryptionkey@0"
I20260809 12:09:35.551770 1336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:09:35.577142 1336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:09:35.581185 1336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:09:35.581478 1336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:09:35.600497 1336 rpc_server.cc:307] RPC server started. Bound to: 127.0.147.124:39889
I20260809 12:09:35.600522 1423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.147.124:39889 every 8 connection(s)
I20260809 12:09:35.601852 1336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/data/info.pb
I20260809 12:09:35.602349 589 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3WUkck/build/asan/bin/kudu as pid 1336
I20260809 12:09:35.602553 589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3WUkck/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786276990256303-589-0/minicluster-data/master-2/wal/instance
I20260809 12:09:35.603214 589 external_mini_cluster.cc:1428] Setting key 6a860c5c00d2b7c8efd873c2e1137bbf
I20260809 12:09:35.606223 1424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:09:35.614812 1424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
I20260809 12:09:35.627622 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.605129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.0.147.124:39889)
0809 12:09:35.606005 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.606017 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.606059 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:09:35.607792 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.607815 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.608192 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.608518 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.610381 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.611800 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.613649 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.614304 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.614425 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.619965 (+ 5540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.619973 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.619983 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.620028 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.623121 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.623933 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.623948 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.623959 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.624142 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.624613 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.624617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.624619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.625110 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.625350 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.625811 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.626262 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":419,"threads_started":1}
I20260809 12:09:35.634492 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.618020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.126:34569 (local address 127.0.0.1:52446)
0809 12:09:35.618783 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.618825 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:35.618974 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:35.619700 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:35.619709 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:35.620478 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:35.620939 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.620956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.622018 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.622029 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.623143 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.623156 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.623986 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.623994 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.624171 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.625462 (+ 1291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:35.625477 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:35.628045 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:35.631697 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.631707 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.631711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.632316 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.632805 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.632811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.632815 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.632984 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.633698 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:35.633704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:35.633854 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 12:09:35.633996 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 12:09:35.634225 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":352,"threads_started":1}
I20260809 12:09:35.634824 1432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.618090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.0.147.126:34569)
0809 12:09:35.619355 (+ 1265us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.619359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.619375 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:09:35.619422 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.619425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.619482 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.619647 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.621111 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.621857 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.623293 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.623784 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.623918 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.628257 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.628263 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.628266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.628300 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.631580 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.632480 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.632485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.632487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.632661 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.633128 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.633132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.633134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.633529 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.633724 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.634131 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.634338 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1152,"thread_start_us":1048,"threads_started":1}
I20260809 12:09:35.636170 1424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.126@KRBTEST.COM
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
I20260809 12:09:35.647666 1433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.630714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52462 (local address 127.0.147.126:34569)
0809 12:09:35.631479 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.631490 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.631515 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:09:35.631594 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.631598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.631675 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.631770 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.633297 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.634447 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.636084 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.636558 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.636714 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.641538 (+ 4824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.641547 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.641553 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.641590 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.644591 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.645294 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.645302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.645307 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.645461 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.645895 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.645902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.645906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.646263 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.646565 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.646945 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.647174 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":334,"threads_started":1}
I20260809 12:09:35.651096 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.636906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37140 (local address 127.0.147.125:33455)
0809 12:09:35.637121 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.637127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.637151 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:09:35.637290 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.637294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.637354 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.637496 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.638122 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.639264 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.640504 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.641250 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.641399 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.644884 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.644889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.644892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.644922 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.648341 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.649081 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.649085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.649088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.649214 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.649688 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.649694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.649697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.650109 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.650263 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.650602 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.650849 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:09:35.651147 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.636788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.125:33455 (local address 127.0.0.1:37140)
0809 12:09:35.637063 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.637077 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:35.637153 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:35.637519 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:35.637522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:35.637840 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:35.637972 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.637979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.639418 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.639422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.640332 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.640368 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.641415 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.641422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.641657 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.642223 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:35.642250 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:35.644709 (+ 2459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:35.648470 (+ 3761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.648473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.648476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.648940 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.649359 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.649364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.649368 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.649551 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.650240 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:35.650244 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:35.650358 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 12:09:35.650467 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 12:09:35.650762 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260809 12:09:35.652191 1424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.147.125@KRBTEST.COM
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.124@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
I20260809 12:09:35.662201 1222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } attempt: 4
I20260809 12:09:35.666296 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.653018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41976)
0809 12:09:35.653262 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.653275 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:35.653351 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:35.653592 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:35.653596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:35.653947 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:35.654074 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.654081 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.655016 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.655024 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.656177 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.656193 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.656914 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.656924 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.657174 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.657766 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:35.657791 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:35.660257 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:35.663744 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.663749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.663752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.664330 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.664707 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.664713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.664716 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.664847 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.665530 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:35.665534 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:35.665662 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 12:09:35.665812 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 12:09:35.666009 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260809 12:09:35.666620 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.653000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.0.147.124:39889)
0809 12:09:35.653194 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.653198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.653334 (+ 136us) server_negotiation.cc:407] Connection header received
0809 12:09:35.653417 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.653420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.653472 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.653577 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.654202 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.654901 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.656327 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.656737 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.656828 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.660421 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.660425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.660428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.660454 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.663590 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.664474 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.664477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.664479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.664596 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.664950 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.664958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.664961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.665392 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.665567 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.665935 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.666195 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":72}
I20260809 12:09:35.667897 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.649584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37142 (local address 127.0.147.125:33455)
0809 12:09:35.650295 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.650300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.650320 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:09:35.650367 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.650370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.650419 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.650543 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.652117 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.652995 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.654546 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.655004 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.655142 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.660288 (+ 5146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.660293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.660296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.660325 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.663374 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.664120 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.664126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.664132 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.664289 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.664798 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.664803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.664806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.665163 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.665368 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.667270 (+ 1902us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.667488 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":333,"threads_started":1}
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.126@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
I20260809 12:09:35.672993 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759: Bootstrap starting.
I20260809 12:09:35.673151 589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:09:35.677093 1424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:35.678556 1424 log.cc:824] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:35.678861 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.662723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:41992)
0809 12:09:35.662887 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.662906 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:35.663004 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:35.664241 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:35.664246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:35.664785 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:35.664997 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.665009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.666715 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.666720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.668128 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.668141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.669188 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.669195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.669346 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.669919 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:35.669947 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:35.672315 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:35.675814 (+ 3499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.675820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.675824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.676415 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.676868 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.676878 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.676882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.677068 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.677774 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:35.677780 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:35.677994 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 12:09:35.678152 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 12:09:35.678445 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 12:09:35.678939 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.663039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41992 (local address 127.0.147.124:39889)
0809 12:09:35.663893 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.663898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.663918 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:09:35.663965 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.663970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.664032 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.664201 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.665164 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.666555 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.668265 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.668866 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.669066 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.672612 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.672619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.672624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.672663 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.675682 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.676553 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.676566 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.676569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.676729 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.677206 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.677211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.677214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.677621 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.677785 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.678282 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.678501 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":713,"thread_start_us":471,"threads_started":1}
I20260809 12:09:35.676223 1191 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e26118b4572847a697ba9ae57e5b006b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52462
I20260809 12:09:35.682368 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759: No bootstrap required, opened a new log
I20260809 12:09:35.684621 1222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b: Bootstrap starting.
I20260809 12:09:35.688433 1222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:35.689670 1222 log.cc:824] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:35.692996 1222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b: No bootstrap required, opened a new log
I20260809 12:09:35.692790 1424 raft_consensus.cc:348] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } }
I20260809 12:09:35.693214 1424 raft_consensus.cc:729] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eee105ad0b9a427dbcbe792d722a8759, State: Initialized, Role: FOLLOWER
I20260809 12:09:35.693747 1424 consensus_queue.cc:260] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } }
I20260809 12:09:35.694856 1439 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.695425 1439 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:35.696108 1424 sys_catalog.cc:564] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:35.701026 1222 raft_consensus.cc:348] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } }
I20260809 12:09:35.701437 1222 raft_consensus.cc:729] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e26118b4572847a697ba9ae57e5b006b, State: Initialized, Role: FOLLOWER
I20260809 12:09:35.702041 1222 consensus_queue.cc:260] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } }
I20260809 12:09:35.704413 1222 sys_catalog.cc:564] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:35.704524 1191 raft_consensus.cc:482] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 12:09:35.704699 1191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:35.708806 1191 raft_consensus.cc:504] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } }
I20260809 12:09:35.711184 1191 leader_election.cc:288] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [CANDIDATE]: Term 1 election: Requested vote from peers 216d220a4ec845c18a9212f5a086f432 (127.0.147.125:33455), eee105ad0b9a427dbcbe792d722a8759 (127.0.147.124:39889)
I20260809 12:09:35.711565 1389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e26118b4572847a697ba9ae57e5b006b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eee105ad0b9a427dbcbe792d722a8759"
I20260809 12:09:35.712076 1389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [term 0 FOLLOWER]: Advancing to term 1
W20260809 12:09:35.714250 1453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:09:35.714392 1453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:09:35.703243 1454 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.714740 1454 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:35.716866 1389 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e26118b4572847a697ba9ae57e5b006b in term 1.
I20260809 12:09:35.717634 1154 leader_election.cc:302] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e26118b4572847a697ba9ae57e5b006b, eee105ad0b9a427dbcbe792d722a8759; no voters:
I20260809 12:09:35.718101 1454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:09:35.703852 1336 master_runner.cc:186] Error getting master registration for 127.0.147.126:34569: OK, instance_id { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" instance_seqno: 1786277374521718 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:09:35.724707 1336 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:09:35.727608 1454 raft_consensus.cc:686] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [term 1 LEADER]: Becoming Leader. State: Replica: e26118b4572847a697ba9ae57e5b006b, State: Running, Role: LEADER
I20260809 12:09:35.728276 1454 consensus_queue.cc:237] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } }
I20260809 12:09:35.736265 1461 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e26118b4572847a697ba9ae57e5b006b. Latest consensus state: current_term: 1 leader_uuid: "e26118b4572847a697ba9ae57e5b006b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.736788 1461 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:35.742280 1476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:09:35.754786 1476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 12:09:35.760805 1166 tablet.cc:2369] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:09:35.772019 1325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } attempt: 5
I20260809 12:09:35.774827 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.745666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.0.147.124:39889)
0809 12:09:35.745872 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.745879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.746134 (+ 255us) server_negotiation.cc:407] Connection header received
0809 12:09:35.746631 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.746640 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.746753 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.746842 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.751266 (+ 4424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.752644 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.756250 (+ 3606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.757042 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.757164 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.765509 (+ 8345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.765516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.765521 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.765576 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.769528 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.770818 (+ 1290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.770824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.770828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.771002 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.771681 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.771699 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.771703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.772065 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.772362 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.773438 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.773869 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:09:35.780119 1492 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
Aug 09 12:09:35 dist-test-slave-sb2z krb5kdc[1110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.147.125@KRBTEST.COM for kudu/127.0.147.124@KRBTEST.COM
I20260809 12:09:35.789083 1433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.744690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52474 (local address 127.0.147.126:34569)
0809 12:09:35.744905 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.744913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.752786 (+ 7873us) server_negotiation.cc:407] Connection header received
0809 12:09:35.754243 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.754251 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.754344 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.754416 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.765112 (+ 10696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.766257 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.772372 (+ 6115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.773030 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.773148 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.780519 (+ 7371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.780525 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.780529 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.780569 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.784963 (+ 4394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.786200 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.786206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.786210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.786650 (+ 440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.787141 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.787149 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.787152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.787568 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.787852 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.788426 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.788699 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 12:09:35.791663 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.772643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.147.124:39889 (local address 127.0.0.1:42000)
0809 12:09:35.773084 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 12:09:35.773108 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:09:35.773219 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:09:35.774240 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:09:35.774245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:09:35.774787 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:09:35.774978 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.774990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.776745 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.776750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.778295 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:09:35.778310 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.779892 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.779899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:09:35.780120 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.780774 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:09:35.780798 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:09:35.784115 (+ 3317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:09:35.788368 (+ 4253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.788374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.788378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.788923 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.789692 (+ 769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:09:35.789697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:09:35.789701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:09:35.789859 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:09:35.790613 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:09:35.790618 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:09:35.790774 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 12:09:35.790921 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 12:09:35.791208 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":1}
I20260809 12:09:35.792650 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.773430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.0.147.124:39889)
0809 12:09:35.773638 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.773644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.773661 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:09:35.773707 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.773712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.773780 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.773880 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.775455 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.776561 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.778462 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.779331 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.779484 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.784470 (+ 4986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.784477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.784481 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.784525 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.788206 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.789091 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.789115 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.789119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.789537 (+ 418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.790005 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.790010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.790014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.790379 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.790545 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.792052 (+ 1507us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.792297 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:09:35.785004 1130 master_runner.cc:186] Error getting master registration for 127.0.147.125:33455: OK, instance_id { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" instance_seqno: 1786277375043103 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:09:35.796020 1130 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:09:35.799018 1325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432: Bootstrap starting.
I20260809 12:09:35.804716 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:35.753468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.0.147.125:33455)
0809 12:09:35.753669 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:35.753675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:35.762214 (+ 8539us) server_negotiation.cc:407] Connection header received
0809 12:09:35.762505 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:35.762510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:35.762600 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:35.762684 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:35.769310 (+ 6626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.770365 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.776868 (+ 6503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:35.778546 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:35.778664 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:35.797437 (+ 18773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:09:35.797444 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:09:35.797448 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:09:35.797484 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:09:35.801246 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.802146 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.802153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.802160 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.802308 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:09:35.802843 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:09:35.802849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:09:35.802855 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:09:35.803241 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:09:35.803497 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:35.804098 (+ 601us) server_negotiation.cc:299] Negotiation successful
0809 12:09:35.804339 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":977,"server-negotiator.queue_time_us":72}
I20260809 12:09:35.807374 1389 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [term 1 FOLLOWER]: Refusing update from remote peer e26118b4572847a697ba9ae57e5b006b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:09:35.808197 1325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432: Neither blocks nor log segments found. Creating new log.
I20260809 12:09:35.808452 1461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:09:35.809734 1325 log.cc:824] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432: Log is configured to *not* fsync() on all Append() calls
I20260809 12:09:35.820634 1325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432: No bootstrap required, opened a new log
W20260809 12:09:35.833626 1363 tablet.cc:2369] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:09:35.838953 1325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } }
I20260809 12:09:35.839462 1325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 216d220a4ec845c18a9212f5a086f432, State: Initialized, Role: FOLLOWER
I20260809 12:09:35.840085 1325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } }
I20260809 12:09:35.843230 1439 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e26118b4572847a697ba9ae57e5b006b. Latest consensus state: current_term: 1 leader_uuid: "e26118b4572847a697ba9ae57e5b006b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.843577 1439 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:35.846336 1499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.846763 1499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:35.848744 1325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:09:35.849176 1285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:09:35.848773 1287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e26118b4572847a697ba9ae57e5b006b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "216d220a4ec845c18a9212f5a086f432"
I20260809 12:09:35.857365 1476 catalog_manager.cc:1347] Generated new cluster ID: cd540f557b994cca92b80b09c5944733
I20260809 12:09:35.857467 1476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:09:35.857869 1454 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e26118b4572847a697ba9ae57e5b006b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.858321 1454 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:35.857355 1285 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [term 1 FOLLOWER]: Refusing update from remote peer e26118b4572847a697ba9ae57e5b006b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:09:35.863703 1287 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e26118b4572847a697ba9ae57e5b006b in term 1.
I20260809 12:09:35.865998 1461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [LEADER]: Connected to new peer: Peer: permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:09:35.898087 1505 mvcc.cc:205] Tried to move back new op lower bound from 7316592131272597504 to 7316592130989170688. Current Snapshot: MvccSnapshot[applied={T|T < 7316592131272597504 or (T in {7316592131272597504})}]
I20260809 12:09:35.898887 1499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e26118b4572847a697ba9ae57e5b006b. Latest consensus state: current_term: 1 leader_uuid: "e26118b4572847a697ba9ae57e5b006b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.900595 1499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:35.900193 1514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e26118b4572847a697ba9ae57e5b006b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.900998 1514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:35.901717 1439 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e26118b4572847a697ba9ae57e5b006b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.901979 1439 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:09:35.922312 1523 catalog_manager.cc:1259] Loaded cluster ID: cd540f557b994cca92b80b09c5944733
I20260809 12:09:35.922429 1523 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432: loading cluster ID for follower catalog manager: success
W20260809 12:09:35.923642 1523 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260809 12:09:35.923799 1523 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:09:35.947904 1461 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e26118b4572847a697ba9ae57e5b006b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e26118b4572847a697ba9ae57e5b006b" member_type: VOTER last_known_addr { host: "127.0.147.126" port: 34569 } } peers { permanent_uuid: "216d220a4ec845c18a9212f5a086f432" member_type: VOTER last_known_addr { host: "127.0.147.125" port: 33455 } } peers { permanent_uuid: "eee105ad0b9a427dbcbe792d722a8759" member_type: VOTER last_known_addr { host: "127.0.147.124" port: 39889 } } }
I20260809 12:09:35.948199 1461 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b [sys.catalog]: This master's current role is: LEADER
I20260809 12:09:35.956770 1476 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:09:35.958724 1476 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:09:35.976281 1476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e26118b4572847a697ba9ae57e5b006b: Generated new TSK 0
I20260809 12:09:35.977447 1476 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:09:36.120803 1430 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.107098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.0.147.124:39889)
0809 12:09:36.107295 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.107302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.118543 (+ 11241us) server_negotiation.cc:407] Connection header received
0809 12:09:36.119037 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.119043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.119161 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.119239 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.120548 (+ 1309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42002: BlockingRecv error: recv got EOF from 127.0.0.1:42002 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 12:09:36.124866 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.122664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37168 (local address 127.0.147.125:33455)
0809 12:09:36.122831 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.122837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.123278 (+ 441us) server_negotiation.cc:407] Connection header received
0809 12:09:36.123518 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.123524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.123638 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.123712 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.124619 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37168: BlockingRecv error: recv got EOF from 127.0.0.1:37168 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 12:09:36.132946 1433 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.115411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52482 (local address 127.0.147.126:34569)
0809 12:09:36.115621 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.115627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.121748 (+ 6121us) server_negotiation.cc:407] Connection header received
0809 12:09:36.121902 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.121909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.122000 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.122158 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.132664 (+ 10506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52482: BlockingRecv error: recv got EOF from 127.0.0.1:52482 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 12:09:36.176177 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.165073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37184 (local address 127.0.147.125:33455)
0809 12:09:36.165259 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.165267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.172446 (+ 7179us) server_negotiation.cc:407] Connection header received
0809 12:09:36.174354 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.174374 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.174583 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.174682 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.175915 (+ 1233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37184: BlockingRecv error: recv got EOF from 127.0.0.1:37184 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 12:09:36.229971 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.217435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.0.147.125:33455)
0809 12:09:36.217625 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.217631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.228143 (+ 10512us) server_negotiation.cc:407] Connection header received
0809 12:09:36.228394 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.228401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.228497 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.228582 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.229696 (+ 1114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37190: BlockingRecv error: recv got EOF from 127.0.0.1:37190 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 12:09:36.293284 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.283858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.0.147.125:33455)
0809 12:09:36.284072 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.284084 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.291262 (+ 7178us) server_negotiation.cc:407] Connection header received
0809 12:09:36.291786 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.291792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.291885 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.291961 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.293054 (+ 1093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37206: BlockingRecv error: recv got EOF from 127.0.0.1:37206 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 12:09:36.361969 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.349708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37212 (local address 127.0.147.125:33455)
0809 12:09:36.349980 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.349987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.360124 (+ 10137us) server_negotiation.cc:407] Connection header received
0809 12:09:36.360416 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.360422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.360536 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.360621 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.361730 (+ 1109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37212: BlockingRecv error: recv got EOF from 127.0.0.1:37212 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
W20260809 12:09:36.379720 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.377412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.0.147.125:33455)
0809 12:09:36.377676 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.377685 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.378259 (+ 574us) server_negotiation.cc:407] Connection header received
0809 12:09:36.378482 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.378486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.378559 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.378671 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.379453 (+ 782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37216: BlockingRecv error: recv got EOF from 127.0.0.1:37216 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 12:09:36.401539 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.399089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.0.147.125:33455)
0809 12:09:36.399321 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.399334 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.399952 (+ 618us) server_negotiation.cc:407] Connection header received
0809 12:09:36.400185 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.400189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.400257 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.400385 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.401192 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37220: BlockingRecv error: recv got EOF from 127.0.0.1:37220 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 12:09:36.440300 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.438271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37222 (local address 127.0.147.125:33455)
0809 12:09:36.438473 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.438479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.438841 (+ 362us) server_negotiation.cc:407] Connection header received
0809 12:09:36.439003 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.439011 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.439128 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.439252 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.440051 (+ 799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37222: BlockingRecv error: recv got EOF from 127.0.0.1:37222 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 12:09:36.509802 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.507909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37232 (local address 127.0.147.125:33455)
0809 12:09:36.508113 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.508120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.508496 (+ 376us) server_negotiation.cc:407] Connection header received
0809 12:09:36.508681 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.508687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.508778 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.508906 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.509621 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37232: BlockingRecv error: recv got EOF from 127.0.0.1:37232 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 12:09:36.648000 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.646270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37248 (local address 127.0.147.125:33455)
0809 12:09:36.646475 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.646482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.646783 (+ 301us) server_negotiation.cc:407] Connection header received
0809 12:09:36.646941 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.646945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.647017 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.647111 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.647806 (+ 695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37248: BlockingRecv error: recv got EOF from 127.0.0.1:37248 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:09:36.716552 1453 catalog_manager.cc:1259] Loaded cluster ID: cd540f557b994cca92b80b09c5944733
I20260809 12:09:36.716706 1453 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759: loading cluster ID for follower catalog manager: success
I20260809 12:09:36.719129 1453 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759: acquiring CA information for follower catalog manager: success
I20260809 12:09:36.720928 1453 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eee105ad0b9a427dbcbe792d722a8759: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 12:09:36.911157 1434 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:09:36.909205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.0.147.125:33455)
0809 12:09:36.909470 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:36.909478 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:36.909709 (+ 231us) server_negotiation.cc:407] Connection header received
0809 12:09:36.909950 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:36.909955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:36.910030 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:36.910141 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:09:36.910920 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37250: BlockingRecv error: recv got EOF from 127.0.0.1:37250 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 12:09:36.926878 1523 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432: acquiring CA information for follower catalog manager: success
I20260809 12:09:36.928730 1523 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 216d220a4ec845c18a9212f5a086f432: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:09:37.433079 1608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:37.426940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37258 (local address 127.0.147.125:33455)
0809 12:09:37.427484 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:37.427491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:37.427687 (+ 196us) server_negotiation.cc:407] Connection header received
0809 12:09:37.427923 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:37.427927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:37.428085 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:37.428221 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:37.428700 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.429431 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.430793 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.431202 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.431339 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:37.431912 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:37.432461 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:37.432650 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:37.432703 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 12:09:37.432785 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":260,"threads_started":1}
I20260809 12:09:37.445309 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:37.439368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.0.147.124:39889)
0809 12:09:37.439896 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:37.439904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:37.439927 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:09:37.440242 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:37.440247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:37.440400 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:37.440599 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:37.441017 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.441715 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.443053 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.443476 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.443636 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:37.444180 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:37.444705 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:37.444880 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:37.444933 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 12:09:37.445016 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":274,"threads_started":1}
I20260809 12:09:37.459944 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:37.451402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42022 (local address 127.0.147.124:39889)
0809 12:09:37.451670 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:37.451678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:37.452585 (+ 907us) server_negotiation.cc:407] Connection header received
0809 12:09:37.452911 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:37.452935 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:37.453055 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:37.453136 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:37.454002 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.455224 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.457749 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.458416 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.458539 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:37.459151 (+ 612us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:37.459277 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:37.459379 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:37.459556 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 12:09:37.459638 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 12:09:37.461265 1608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:37.452847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.0.147.125:33455)
0809 12:09:37.453068 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:37.453076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:37.454890 (+ 1814us) server_negotiation.cc:407] Connection header received
0809 12:09:37.455086 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:37.455091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:37.455159 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:37.455332 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:37.456369 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.457203 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.459167 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.459837 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.459962 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:37.460485 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:37.460635 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:37.460722 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:37.460869 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 12:09:37.460973 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:09:37.462291 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:09:37.451217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52496 (local address 127.0.147.126:34569)
0809 12:09:37.452046 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0809 12:09:37.452055 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:09:37.454713 (+ 2658us) server_negotiation.cc:407] Connection header received
0809 12:09:37.454862 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:09:37.454868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:09:37.455084 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:09:37.455264 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:09:37.456440 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.457564 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.459724 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:09:37.460476 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:09:37.460603 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:09:37.461179 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:09:37.461666 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:09:37.461783 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 12:09:37.461912 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 12:09:37.461996 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":517,"threads_started":1}
I20260809 12:09:37.470949 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 1130
I20260809 12:09:37.472146 1214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:37.584154 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 1130
I20260809 12:09:37.613646 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 1230
I20260809 12:09:37.614804 1320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:37.724422 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 1230
I20260809 12:09:37.755615 589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3WUkck/build/asan/bin/kudu pid 1336
I20260809 12:09:37.756906 1416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:09:37.874909 589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3WUkck/build/asan/bin/kudu with pid 1336
2026-08-09T12:09:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5479 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10804 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (387543 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST